应对网络 复杂性

Exploring the Problem Space

第 4 章：操作复杂性

Chapter 4: Operational Complexity

探索问题空间

人类与系统交互的成本

The Cost of Human Interaction with the System

策略分散与最佳流量处理

Policy Dispersion versus Optimal Traffic Handling

解决管理复杂性问题

Solving the Management Complexity Problem

自动化作为管理复杂性的解决方案

Automation as a Solution to Management Complexity

模块化作为管理复杂性的解决方案

Modularity as a Solution to Management Complexity

协议复杂性与管理复杂性

Protocol Complexity versus Management Complexity

Control Plane State versus Stretch

第 5 章：设计复杂性

Chapter 5: Design Complexity

控制平面状态与拉伸

聚合与拉伸

Aggregation versus Stretch

交通工程

Traffic Engineering

状态与拉伸：一些最终想法

State versus Stretch: Some Final Thoughts

拓扑与收敛速度

Topology versus Speed of Convergence

环形拓扑收敛

Ring Topology Convergence

冗余与弹性

Redundancy versus Resilience

拓扑与收敛速度：一些最终的想法

Topology versus Speed of Convergence: Some Final Thoughts

快速收敛与复杂性

Fast Convergence versus Complexity

通过智能定时器提高融合：说话速度更快

Improving Convergence with Intelligent Timers: Talk Faster

从收敛中删除计时器：预计算

Removing Timers from Convergence: Precompute

解决拓扑问题：通过隧道连接至无环路备用网络

Working around Topology: Tunneling to the Loop-Free Alternate

关于快速收敛的一些最终想法

Some Final Thoughts on Fast Convergence

虚拟化与设计复杂性

Virtualization versus Design Complexity

功能分离

Functional Separation

转发平面复杂度

Forwarding Plane Complexity

控制平面复杂性

Control Plane Complexity

命运共同体风险群体

Shared Fate Risk Groups

Chapter 6: Managing Design Complexity

第 6 章：管理设计复杂性

Interchangeable Modules

模块化如何解决复杂性问题

How Modularity Attacks the Complexity Problem

Failure Domains and Information Hiding

关于信息隐藏的最终想法

Final Thoughts on Information Hiding

Places in the Network

Chapter 7: Protocol Complexity

灵活性与复杂性：OSPF 与 IS-IS

Flexibility versus Complexity: OSPF versus IS-IS

分层与协议复杂性

Layering versus Protocol Complexity

七层模型

The Seven-Layer Model

Protocol Stacks and Design

协议复杂性与设计复杂性

Protocol Complexity versus Design Complexity

微循环和快速重新路由

Microloops and Fast Reroute

EIGRP 和设计难题

EIGRP and the Design Conundrum

关于协议复杂性的最终想法

Final Thoughts on Protocol Complexity

第 8 章：复杂系统如何失败

Chapter 8: How Complex Systems Fail

反馈回路

Feedback Loops

网络工程中的正反馈循环

Positive Feedback Loops in Network Engineering

速度、状态和表面：网络控制平面的稳定性

Speed, State, and Surface: Stability in the Network Control Plane

TCP Synchronization as a Shared Fate Problem

Thoughts on Root Cause Analysis

根本原因分析的思考

工程技能和故障管理

Engineering Skills and Failure Management

第 9 章：可编程网络

Chapter 9: Programmable Networks

驱动因素和定义

Drivers and Definition

业务驱动因素

Business Drivers

中心化的潮起潮落

The Ebb and Flow of Centralization

定义网络可编程性

Defining Network Programmability

可编程网络的用例

Use Cases for Programmable Networks

带宽日历

Bandwidth Calendaring

软件定义的边界

Software-Defined Perimeter

可编程网络接口

Programmable Network Interfaces

可编程网络格局

The Programmable Network Landscape

Path Computation Element Protocol

路由系统接口

Interface to the Routing System

Chapter 10: Programmable Network Complexity

第 10 章：可编程网络复杂性

辅助原则

The Subsidiarity Principle

Surface and the Programmable Network

对故障域的影响

Impact on Failure Domains

广域故障域

Wide Area Failure Domains

数据中心故障域

Data Center Failure Domains

控制平面故障域的应用

Application to Control Plane Failure Domain

控制器到控制器故障域

Controller to Controller Failure Domain

关于故障域的最终想法

Final Thoughts on Failure Domains

Chapter 11: Service Virtualization and Service Chaining

第 11 章：服务虚拟化和服务链

网络功能虚拟化

Network Function Virtualization

Service Function Chaining

Chapter 12: Virtualization and Complexity

策略分散和网络虚拟化

Policy Dispersion and Network Virtualization

状态和服务链

State and Service Chaining

状态和优化

State and Optimization

表面与政策互动

Surface and Policy Interaction

表面和策略代理

Surface and Policy Proxies

其他设计考虑因素

Other Design Considerations

耦合和失效域

Coupling and Failure Domains

故障排除

Troubleshooting

编排效果

The Orchestration Effect

Chapter 13: Complexity and the Cloud

复杂性体现在哪里？

Where Does the Complexity Live?

Is There a “Right Way?”

Chapter 14: A Simple Ending

定义复杂性

Defining Complexity

难以理解

Difficult to Understand

意想不到的后果

Unintended Consequences

大量相互作用的部件

Large Numbers of Interacting Parts

是什么让事情“太复杂”？

What Makes Something “Too Complex”?

复杂性是一种权衡

Complexity Is a Tradeoff

建模复杂性

Modeling Complexity

管理现实世界中的复杂性

Managing Complexity in the Real World

不要忽视复杂性

Don’t Ignore Complexity

找到一个模型来控制复杂性

Find a Model to Contain the Complexity

指数

Index

介绍

Introduction

每个工程师，无论他们从事什么类型的工程，几乎总是面临着复杂性。更快、更便宜、更高质量的三合一是工程师所做的一切的不变伴侣。但有时，复杂性并不那么明显。例如，软件项目需要多长时间？常见的回答是两个小时、两周或太长——又一次遭遇复杂性。

Every engineer, no matter what type of engineering they do, face complexity almost constantly. The faster, cheaper, higher quality triad are the constant companion of everything the engineer does. Sometimes, though, the complexity isn’t so obvious. For instance, how long will the software project take? Two hours, two weeks, or too long is the common reply—another encounter with complexity.

虽然复杂性理论的研究在科学和数学领域进展迅速，但这些理论和思想在更实际的工程问题中的应用，特别是以“普通工程师”可以阅读和理解的方式，根本没有跟上步伐。本书旨在填补网络工程的这一空白。

While research into complexity theory has proceeded apace in the scientific and mathematical worlds, the application of the theories and ideas to more practical engineering problems, particularly in a way that the “average engineer” can read and understand, simply hasn’t kept pace. This book aims to fill that gap for network engineering.

这本书缺乏复杂性理论家正在研究的优雅的数学模型，这可能会让拥有数学学位的读者感到失望。相反，重点是复杂性理论结构的实际应用。本书不关注“纯理论”和数学，而是关注复杂性理论家正在研究的想法的实际应用。

In a move that will probably be disappointing to readers with a math degree, this book is devoid of the elegant mathematical models complexity theorists are working with. Rather, the focus is on the practical application of the theoretical constructs of complexity. Instead of being focused on the “pure theory” and math, this book is focused on the practical application of the ideas being investigated by complexity theorists.

因此，本书的目标读者是“普通”网络工程师，他们希望了解为什么特定的常见结构会以它们的方式工作，例如分层设计、聚合和协议分层。通过深入了解这些被广泛接受的网络设计方法，揭示每种情况下系统中设计的权衡原因，作者希望读者学会吸取在一个领域学到的经验教训并将其应用到其他领域。读完本书后，网络工程师应该开始理解为什么分层设计和分层等能够发挥作用，以及如何在更具体的情况下查看和更改权衡。

This book, then, is targeted at the “average” network engineer who wants to gain an understanding of why particular common constructs work the way they do, such as hierarchical design, aggregation, and protocol layering. By getting behind these widely accepted ways of designing a network, exposing the reasons for the tradeoffs designed into the system in each case, the authors hope the reader learns to take lessons learned in one area and apply them to other areas. After reading this book, network engineers should begin to understand why hierarchical design and layering, for instance, work, and how to see and change the tradeoffs in more specific situations.

本书的结构

How This Book Is Organized

本书在第一章中从复杂性理论的高层次观点开始。这不是深入的理论，而是旨在提供实践视图无需深入研究繁重的数学（或任何数学）即可了解复杂性。第二章概述了测量网络复杂性的各种尝试，包括每次尝试中遇到的一些问题。第三章提出了一个复杂性模型，该模型将在本书的其余部分中使用。

This book begins, in the first chapter, with a high level view of complexity theory. This is not deep theory, but it is rather designed to provide a hands-on view of complexity without diving into heavy math (or any math at all). The second chapter provides an overview of various attempts at measuring complexity in a network, including some of the problems plauging each attempt. The third chapter proposes a model of complexity that will be used throughout the rest of the book.

第 4 章到第 7 章考虑了网络和协议设计的复杂性示例。这些章节试图阐明的要点是前三章中的模型和概念是如何实现的。第 8 章是进入网络工程术语中的复杂系统故障世界的重要途径。本书的其余部分首先概述了网络工程师在撰写本文时面临的三种新技术，然后从复杂性权衡方面对每一种技术进行了分析。

Chapter 4 through 7 consider examples of complexity in network and protocol design. The point these chapters attempt to drive home is how the models and concepts in the first three chapters. Chapter 8 is something of an important detour into the world of complex system failure in network engineering terms. The rest of the book is dedicated to first providing an overview of three new technologies network engineers face at the time of writing, and then an analysis of each one in terms of complexity tradeoffs.

请注意，以 Draft-xxx 形式引用的文档是 IETF 正在进行的工作，因此没有稳定的文档名称或统一资源定位符 (URL)。由于这些限制，没有给出明确的参考文献，而只是给出了文档的标题和文档名称。要查找这些文档，请在 IETF 网站 ( ietf.org )上执行文档搜索。

Note, Documents referenced with the form draft-xxx are IETF works in progress, and therefore do not have stable document names or uniform resource locators (URLs). Because of these limitations, explicit references have not been given, but rather just the title of the document and the document name. To find these documents, please perform a document search at the IETF website (ietf.org).

第 1 章定义复杂性

Chapter 1. Defining Complexity

计算机网络很复杂。

Computer networks are complex.

但“计算机网络很复杂”是什么意思呢？你能把一个网络放在一个秤上并让指针指向“复杂”吗？是否有一个数学模型可以将一组网络设备的配置和拓扑插入其中，从而产生“复杂性指数”？规模、弹性、脆弱性和优雅等概念与复杂性有何关系？不幸的是，这些问题的答案很复杂。事实上，回答这些问题最困难的问题是决定从哪里开始。

But what does “computer networks are complex” mean? Can you put a network on a scale and have the needle point to “complex”? Is there a mathematical model into which you can plug the configurations and topology of a set of network devices that will, in turn, produce a “complexity index”? How do the concepts of scale, resilience, brittleness, and elegance, relate to complexity? The answers to these questions are—unfortunately—complex. In fact, the most difficult issue involved in answering these questions is deciding where to begin.

最好的起点是从头开始——在这种情况下，复杂性的一些定义，从“我不理解的一切”到“涉及很多无意后果的事情”。这些答案至少都有一定的道理。每个答案的某些部分都有助于构建总体复杂性以及网络设计和架构的复杂性的图景。

The best place to begin is at the beginning—in this case a few definitions of complexity, from “everything I don’t understand,” to “that which involves a lot of unintentional consequences.” There is at least some truth in each of these answers; some part of each of these answers is helpful in building a picture of complexity in general, and complexity in network design and architecture.

一旦审视了复杂性的含义，本书将转而探讨为什么计算机网络首先必须是复杂的。如果工程师避免了从协议设计到网络管理的所有复杂性，事情不是会简单得多吗？难道不能以某种方式“管理”复杂性吗？本章第二节概述了复杂性领域的研究。事实证明，复杂性是现实世界中必要的权衡。要达到这一目标，需要通过一些相当——请原谅双关语——复杂的材料。有必要研究复杂性的组成部分，并将一个广泛的想法分解为一组实际上可以以有用的方式理解的组成部分。

Once the meaning of complexity has been examined, this book will turn to asking why computer networks must be complex in the first place. Wouldn’t things be a lot simpler if engineers just avoided all complexity from protocol design to network management? Can’t complexity be “managed out” in some way? The second section of this chapter provides an overview of research in the field of complexity. As it turns out, complexity is a necessary tradeoff in the real world. Getting there will require winding through some rather—pardon the pun—complex material. It’s going to be necessary to look at the components of complexity and to dissect a broad idea into a set of components that can actually be understood in a useful way.

第 2 章“复杂性的组成部分”将开始研究复杂性与网络工程之间的交叉点，考虑各种因素对复杂性的反应。网络工程师往往对复杂性有五种反应之一，其中三种可以形成积极的反应，两种通常是破坏性的。三种积极回应如下：

Chapter 2, “Components of Complexity,” will begin investigating the intersection between complexity and network engineering, considering various reactions to complexity. Network engineers tend to have one of five reactions to complexity, three of which can be shaped into positive responses and two are generally destructive. The three positive responses are as follows:

1.抽象掉复杂性，围绕系统的每个部分构建一个黑匣子，以便更容易理解每个部分以及这些部分之间的交互。

1. Abstract the complexity away, to build a black box around each part of the system, so each piece and the interactions between these pieces are more immediately understandable.

2.将复杂性抛到隔间墙上——将问题从网络领域转移到应用程序、编码或协议领域。正如 RFC1925 所说，“转移问题（例如，将问题转移到整个网络架构的不同部分）比解决问题更容易。” ¹

2. Toss the complexity over the cubicle wall—to move the problem out of the networking realm into the realm of applications, or coding, or a protocol. As RFC1925 says, “It is easier to move a problem around (e.g., by moving the problem to a different part of the overall network architecture) than it is to solve it.”¹

1 . Ross Callon 主编，“十二个网络真理”（IETF，1996 年 4 月），https://www.rfc-editor.org/rfc/rfc1925.txt。

1. Ross Callon, ed., “The Twelve Networking Truths” (IETF, April 1996), https://www.rfc-editor.org/rfc/rfc1925.txt.

3.在顶部添加另一层，通过在已有协议或隧道之上放置另一个协议或隧道，将所有复杂性视为黑匣子。回到 RFC1925，“总是可以添加另一个间接级别。” ²

3. Add another layer on top, to treat all the complexity as a black box by putting another protocol or tunnel on top of what’s already there. Returning to RFC1925, “It is always possible to add another level of indirection.”²

2 . 同上。

2. Ibid.

两种普遍否定的反应如下：

The two generally negative responses are as follows:

1.对复杂性感到不知所措，将现有的东西贴上“遗留”的标签，并追逐一些新的闪亮的东西，这些东西将以一种被认为不那么复杂的方式解决所有问题。

1. Become overwhelmed with the complexity, label what exists as “legacy,” and chase some new shiny thing that will solve all the problems in what is perceived as a much less complex way.

2.忽略问题并希望它会消失。一个很好的例子是，主张“仅此一次”例外，以便在非常紧张的时间表内实现特定的业务目标或解决某些问题，并承诺“稍后”处理复杂性问题。

2. Ignoring the problem and hoping it will go away. Arguing for an exception “just this once,” so a particular business goal can be met, or some problem fixed, within a very tight schedule, with the promise that the complexity issue will be dealt with “later,” is a good example.

这些反应表现为网络设计许多不同领域的复杂性的解决方案，包括操作复杂性、设计复杂性和协议复杂性；这些领域中的每一个领域都将在单独的章节中进行更详细的探讨，以便更好地理解增加每个领域的复杂性的成本和收益。

These reactions show up as solutions to complexity in many different realms of network design, including operational complexity, design complexity, and protocol complexity; each of these areas will be examined in more detail in individual chapters to gain a better understanding of the costs and benefits of adding complexity in each one.

一旦涵盖了所有这些背景材料，就该转向一些更实际的示例了。不同技术背后的基本操作和/或概念，有望帮助驯服我们网络中的复杂性怪物将予以考虑，包括检查每一项在何处减少和增加了复杂性。

Once all this background material is covered, it will be time to turn to some more practical examples. The basic operation and/or concepts behind different technologies that promise to help tame the complexity monster in our networks will be considered, including an examination of where each one reduces and increases complexity.

最后一章将把所有这些结合在一起。最终目标是建立网络系统复杂性的通用视图，可用于做出有关操作、设计和协议的实际决策。虽然从这里到那里需要一些理论，但最终目标迫在眉睫：学习如何识别和管理现实计算机网络世界中的复杂性。

The final chapter will bring it all together. The final goal is to build a generalized view of complexity in network systems that can be applied to making real decisions about operations, design, and protocols. While it will take some theory to get from here to there, the final goal is imminently practical: to learn how to recognize and mange complexity in the real world of computer networks.

什么是复杂性？

What Is Complexity?

当面对“什么是复杂性”这个问题时，您可能会回答：“复杂性就像情人眼里出西施一样。” 然而，接受这个定义比较困难，因为它内化了复杂性。将复杂性变成一种内在的心态或印象，就会使复杂性失去相应的现实，从而使工程师没有工具或模型来理解、控制或管理复杂性。另一方面，面对复杂性的感知具有真正的价值，因为这种感知可以帮助您看到并理解网络中复杂性的现实。

When confronted with the question, “what is complexity,” you might reply, “Complexity—like beauty—is in the eye of the beholder.” It’s harder to live with this definition, however, because it internalizes complexity. Making complexity into an internal state of mind, or an impression, leaves complexity with no corresponding reality, and hence leaves engineers with no tools or models with which to understand, control, or manage complexity. On the other hand, there is real value in confronting the perception of complexity, as the perception can help you see, and understand, the reality of complexity in networks.

一个有用的起点是用两个广泛的观念来定义复杂性：复杂性是我不理解的任何东西，而复杂性是具有大量（移动）部分的任何东西。除了这些之外，重要的是要考虑复杂性作为状态与意图，最后，复杂性和意外后果的规律。

A useful place to begin is defining complexity with two broad perceptions: complexity is anything I don’t understand, and complexity is anything with a lot of (moving) parts. Moving beyond these, it’s important to consider complexity as state versus intent, and then, finally, complexity and the law of unintended consequences.

有什么我不明白的地方

Anything I Don’t Understand

对于一个人来说复杂的事情对于另一个人来说可能很简单。或者，正如克拉克第三定律所说，“任何足够先进的技术都与魔法没有区别。” ³从这些观察中，您可以得出结论：复杂性就是您不理解的任何事物。从编码员的角度来看，一些定义说明了这种心态：

What is complex for one person might be simple for another. Or, as Clarke’s third law says, “Any sufficiently advanced technology is indistinguishable from magic.”³ From these observations, you could conclude that complexity is anything you don’t understand. A few definitions from a coder’s point of view illustrate this state of mind:

3 . 亚瑟·克拉克 (Arthur C. Clarke)，《未来概况：可能性极限的探究》（伦敦：菲尼克斯，2000 年），np

3. Arthur C. Clarke, Profiles of the Future: An Inquiry into the Limits of the Possible (London: Phoenix, 2000), n.p.

• 干净的解决方案是有效且我理解的解决方案。

• A clean solution is a solution that works, and that I understand.

• 复杂的解决方案是指可行但我不理解的解决方案。

• A complex solution is a solution that works, and that I don’t understand.

• 模糊代码是我不理解且没有注释的代码。

• Obscure code is code that I don’t understand, and isn’t commented.

• 自文档化代码是我编写的代码，因此我无需注释即可理解。

• Self-documenting code is code that I wrote, and therefore I can understand without comments.

• hack 是一段不起作用的代码，不是我写的，我也不理解。

• A hack is a piece of code that doesn’t work, I didn’t write, and I don’t understand.

• 临时解决方法是我确实编写并且理解的一段代码。

• A temporary workaround is a piece of code that I did write and I do understand.

但“任何我不明白的事情”在现实世界中是可行的定义吗？有几点应该说明以此作为复杂性的最终定义的谬误：

But is “anything I don’t understand” a workable definition in the real world? Several points should illustrate the fallacy of settling on this as a final definition of complexity:

• 现实世界中有许多复杂的系统，没有人能够完全理解。事实上，现实世界中有许多网络没有人能够完全理解。例如，网络运营商可能了解互联网的一个特定部分，或者了解互联网实际工作原理的大致轮廓，但如果有人声称他们知道互联网的每个部分和各个部分是如何工作的很荒谬。

• There are a number of complex systems in the real world that no one understands in their entirety. In fact, there are a number of networks in the real world that no one understands in their entirety. It might, for instance, be possible for a network operator to understand one specific piece of the Internet, or a general outline of how the Internet really works, but for someone to claim that they know how every part and piece of the Internet works would be absurd.

• 如果这个定义成立的话，加深理解必然意味着降低复杂性。对于任何给定的现象，一旦用纯粹唯物主义的术语“理解”它，就可以宣称它“并不复杂”。例如，在 20 世纪 00 年代初，没有人了解眼睛等生物系统的复杂运作。从那时起，人们就开始研究和记录眼睛的运作（至少在某种程度上）。这是否意味着眼睛现在不像以前那么复杂了？

• Increased understanding necessarily means reduced complexity if this definition is true. For any given phenomenon, once it is “understood” in purely materialistic terms, it can be declared “not complex.” For instance, no one in the early 1900s understood the complex operation of a biological system such as the eye. Since that time, the operation of the eye has been researched and documented (at least to some degree). Does this mean the eye is now less complex than it once was?

• 同样，了解设计者的意图可能会使设计看起来不那么复杂，而无需实际更改设计本身。例如，在协议中使用类型-长度-值 (TLV) 结构似乎会增加整体复杂性，但几乎没有什么好处，但是当考虑到针对新功能的协议的未来验证时，使用更复杂的结构可能是完全有意义的。

• In the same way, understanding the intent of a designer might make a design appear less complex without actually changing the design itself. For instance, using Type-Length-Value (TLV) constructions in a protocol might seem to increase the overall complexity for very little gain—but when future proofing a protocol against new functionality is taken into account, using more complex structures might make perfect sense.

简而言之，复杂性和理解之间的联系充其量是微弱的——更好地理解任何给定的系统并不会真正降低它的复杂性，只会让它更容易理解。不过，不要低估这个定义的基本前提。为什么更好地理解系统会使其显得不那么复杂？几点可以在这里提到，其中每一个都将在本书的其余部分得到更全面的发展：

In short, the connection between complexity and understanding is tenuous at best—gaining a better understanding of any given system doesn’t really make it less complex, it just makes it more understandable. Don’t underestimate the underlying premise of this definition, though. Why does gaining a better understanding of a system make it appear less complex? Several points can be mentioned here, each of which will be developed more fully through the rest of this book:

• 了解系统如何工作可以让您理解并解释各个部分之间的相互作用。那么，您可以用来管理复杂性的步骤之一就是发现系统各部分之间的交互。

• Understanding how a system works allows you to comprehend and account for the interactions between the various parts. One of the steps you can use to manage complexity, then, is to discover the interactions between the parts of the system.

• 了解系统的工作原理使您能够根据一组给定的输入来预测输出，或者更确切地说，根据输入构建系统输出的心理模型。管理复杂性的另一个步骤是将任何给定的系统抽象为一组输入和输出，这些输入和输出可以用作更大上下文（或系统）中该系统的代理。

• Understanding how a system works allows you to predict the output based on a set of given inputs, or rather to construct a mental model of the system’s output based on inputs. Another step you can use to manage complexity is to abstract any given system into a set of inputs and outputs that can be used as a proxy for that system within a larger context (or system).

• 系统的心智模型还可以让您识别系统何时无法正常运行，并解决这些计划外的输出、发现系统为何如此运行并修复故障，或确定输入为何不正常运行。不符合您的期望（并纠正它们）。

• A mental model of the system also allows you to recognize when the system isn’t operating correctly, and to work around these unplanned outputs, discover why the system is acting the way it is and repair the fault, or determine why the inputs don’t match your expectations (and correct them).

虽然将复杂性定义为“我不理解的任何事情”并不是一个完整的定义，但它确实有助于在一些非常重要的方面界定定义的范围。探索被认为“难以理解”的事物可以带来更全面的定义——一个在现实世界中更有用的定义。

While defining complexity as “anything I don’t understand” isn’t a full definition, it does help scope the definition in some very important ways. Exploring things perceived as being “hard to understand” can lead to a fuller definition—a definition that will be more useful in the real world.

任何有很多零件的东西

Anything with a Lot of Parts

在设计大规模解决方案时，一个反复出现的问题是“使用尽可能少的移动部件”。一般来说，任何具有许多部分或许多不同部分的事物都被视为复杂的，从大规模网络到复杂的晶体结构。但是零件的数量，或者移动零件的数量，甚至不同零件的数量，真的可以代表复杂性吗？不必要。两个具体例子反驳了“移动部件的数量是复杂性的良好决定因素”这一观点。

In designing large-scale solutions, one recurring refrain is to “use the minimum possible number of moving parts.” In general, anything with a lot of parts or lots of different parts is seen as complex, from large-scale networks to complex crystal structures. But is the number of parts, or the number of moving parts, or even the number of different parts, really a proxy for complexity? Not necessarily. Two specific examples provide counters to the idea that the number of moving parts is a good determinate of complexity.

第一个如图 1.1所示——以高分辨率拍摄的三维曼德尔球分形。⁴

The first of these is illustrated in Figure 1.1—a three-dimensional Mandelbulb fractal taken at a high resolution.⁴

4 . “文件：Mandelbulb140a.JPG — 维基共享资源”，np，2014 年 7 月 8 日访问，https://commons.wikimedia.org/wiki/File :Mandelbulb140a.JPG 。

4. “File:Mandelbulb140a.JPG—Wikimedia Commons,” n.p., accessed July 8, 2014, https://commons.wikimedia.org/wiki/File:Mandelbulb140a.JPG.

图 1.1 三维曼德尔球

Figure 1.1 A Three-Dimensional Mandelbulb

虽然插图中有很多明显的复杂性，但它实际上都是通过迭代运行相同的算法创建的 - 因此，实际上，有只有一个“移动的部分”。第二个例子来自网络世界，是两种路由协议的交互，例如开放最短路径优先 (OSPF) 和边界网关协议 (BGP)。虽然这两种协议的设计和构造都相当简单，但在大规模网络中部署任何一种协议实际上都涉及很大的复杂性。在 OSPF 之上分层 BGP 所增加的复杂性比不熟悉网络设计的人想象的要大得多。这两个协议之间存在许多意外的交互，从计算最佳路径到确定用于特定目的地的下一跳。它们在聚合时的交互（响应网络拓扑的变化）可能特别有趣。

While there is a lot of apparent complexity in the illustration, it is all actually created by iteratively running the same algorithm—so, in reality, there is only one “moving piece.” A second example, from the networking world, is the interaction of two routing protocols, such as Open Shortest Path First (OSPF) and the Border Gateway Protocol (BGP). While both of these protocols are fairly simple in their design and construction, deploying either one in a large-scale network actually involves a good deal of complexity. Layering BGP on top of OSPF adds a much larger amount of complexity than someone unfamiliar with network design might expect. There are a number of unexpected interactions between these two protocols, from calculating the best path to determining the next hop to use toward a specific destination. Their interaction while converging (in reaction to a change in the network topology) can be especially entertaining.

复杂的模式可以由简单的规则集产生，这应该提醒您不要将复杂性的外观与复杂性本身相混淆。仅仅因为某件事看起来复杂并不意味着它实际上很复杂；相反，你必须超越表象才能真正理解复杂性。某物有很多活动部件的东西不一定很复杂，就像只有几个活动部件的东西不一定很简单一样。真正重要的是运动部件之间的相互作用。

That complex patterns can result from simple rule sets should remind you not to confuse the appearance of complexity with complexity itself. Simply because something looks complex does not mean it actually is complex; instead you must reach beyond appearances to get a true understanding of complexity. Something with a lot of moving parts isn’t necessarily complex, any more than something with just a few moving parts is necessarily simple. What really matters is the interaction between the moving parts.

• 系统的每个部分与其他部分交互的次数越多，并且系统的任何给定部分交互的部分越多，系统本身就会越复杂。系统各部分之间交互的数量可以称为交互平面；交互平面越大，系统各部分之间的交互就越多。同样，交互平面越小，系统各部分之间的交互就越少。

• The more often each piece of a system interacts with others, and the more pieces any given piece of a system interacts with, the more complex the system itself will be. The number of interactions between the various parts of a system can be called the interaction plane; the larger the plane of interaction, the more the parts of the system interact with one another. Likewise, the smaller the plane of interaction, the less the parts of the system interact with one another.

• 系统各部分之间的关系越深，系统就越复杂。

• The deeper the relationship between the various parts of a system, the more complex the system will be.

图 1.2以更直观的方式说明了这些概念。

Figure 1.2 illustrates these concepts in a more visual way.

图 1.2 交互面和深度图示

Figure 1.2 Illustration of Interaction Surface and Depth

举一个更具体的例子，考虑一个设计为跨网络运行的应用程序。按照复杂性增加的顺序提供了四种可能性，与图 1.2中的插图平行：

To give a more concrete example, consider an application designed to run across a network. Four possibilities are provided in order of increasing complexity, parallel to the illustration in Figure 1.2:

• 应用程序的一个实例使用传输控制协议(TCP) 传输信息块，而第二个实例则接收该信息。该应用程序依赖于 TCP 和底层网络来准确无误地传递这些信息块，但不拥有有关 TCP 进程当前状态或实现的任何信息。这将是一个浅而窄的交互表面；应用程序在多个地方与 TCP 交互，但对 TCP 的内部状态知之甚少。

• One instance of the application transmits blocks of information using the Transmission Control Protocol (TCP), while a second instance receives this information. The application relies on TCP and the underlying network to deliver these blocks of information without error, but doesn’t possess any information about the TCP process’ current state or implementation. This would be a shallow, narrow interaction surface; the application interacts with TCP in multiple places, but knows little about TCP’s internal state.

• 应用程序的一个实例使用 TCP 传输信息块，而第二个实例则接收该信息。由于传输的信息有时可能对时间敏感，因此应用程序会不时检查 TCP 队列的状态，调整发送块的速率，并且可以请求 TCP 进程推送块，而不是等待TCP 缓冲区已填满。这是一个更深层次的交互界面，因为应用程序现在必须了解 TCP 的工作方式，并与 TCP 的工作方式进行智能交互。然而，交互面仍然很窄，因为它仅存在于 TCP 和应用程序之间的多个点上。

• One instance of the application transmits blocks of information using TCP, while a second instance receives this information. Because the information being transmitted can sometimes be time sensitive, the application examines the state of the TCP queue from time to time, adjusting the rate at which it is sending blocks, and can request the TCP process to push a block, rather than waiting for the TCP buffer to fill up. This is a deeper interaction surface because the application must now understand something about the way TCP works, and interact with how TCP works intelligently. The interaction surface is still narrow, however, as it is only between TCP and the application at multiple points.

• 应用程序的一个实例使用 TCP 传输信息块，而第二个实例则接收该信息。与第一个示例一样，应用程序不会以任何方式与 TCP 状态进行交互；它只是将数据块放入 TCP 的缓冲区中，并假设 TCP 将通过网络正确传送数据。然而，该应用程序对时间敏感，因此还会定期读取时钟，而时钟的准确性取决于网络时间协议 (NTP)。这个交互面可以被认为更广泛一些，因为应用程序现在正在与两种不同的网络协议交互，但两种交互都相当浅。

• One instance of the application transmits blocks of information using TCP, while a second instance receives this information. As in the first example, the application doesn’t interact with TCP’s state in any way; it simply places blocks of data into TCP’s buffer and assumes TCP will deliver it correctly across the network. The application is, however, time sensitive, and therefore also reads the clock, which is dependent on Network Time Protocol (NTP) for accuracy, on a regular basis. This interaction surface can be considered a bit broader, as the application is now interacting with two different network protocols, but both interactions are rather shallow.

• 应用程序的一个实例使用 TCP 传输信息块，而第二个实例则接收该信息。由于传输的信息对时间敏感，因此应用程序会不时检查 TCP 队列的状态，调整发送块的速率，并且可以请求 TCP 进程推送块，而不是等待 TCP缓冲区填满。为了对这些推送请求进行计时，应用程序会监视本地时间，并假设使用 NTP（或某种类似的解决方案）在多个设备之间保持同步。此示例中的交互面既广泛又深入，因为应用程序再次需要了解 TCP 数据传输的内部工作原理。它也更广泛，因为应用程序正在与两个协议（或更大意义上的系统）交互，而不是一个。事实上，在这个例子中，NTP 的状态现在通过应用程序与 TCP 的内部状态绑定在一起，并且两个协议都不知道这一点联系。

• One instance of the application transmits blocks of information using TCP, while a second instance receives this information. Because the information being transmitted is time sensitive, the application examines the state of the TCP queue from time to time, adjusting the rate at which it is sending blocks, and can request the TCP process to push a block, rather than waiting for the TCP buffer to fill up. To time these push requests, the application monitors the local time, which it assumes is kept synchronized among multiple devices using NTP (or some similar solution). The interaction surface in this example is both broader and deeper, in that the application again needs to know the inner workings of TCP’s data transmission. It is also broader, in that the application is interacting with two protocols (or systems, in the larger sense), rather than one. In fact, in this example, the state of NTP is now tied to the internal state of TCP through the application—and neither protocol knows about this connection.

那么，交互面就是两个系统交互的位置数量以及这些交互的深度；每个附加的应用程序编程接口 (API)、套接字或其他接触点都会增加交互表面。很明显，随着系统内两个子系统之间交互的数量和级别的增加，复杂性级别也会提高。

The interaction surface, then, is simply the number of places where two systems interact and the depth of those interactions; each additional Application Programming Interface (API), socket, or other point of contact increases the interaction surface. It’s clear that as the amount and level of interaction between two subsystems within a system increases, the complexity level is driven up.

这些概念将与网络设计和体系结构更全面地相关，因为本书将通过各种示例进行介绍，但请考虑一下：例如，OSPF 和中间系统到中间系统 (IS-IS) 作为“夜间船舶”运行路由协议。它们都依赖于有关 IP 地址、链路状态、指标以及直接从网络拓扑中提取的其他信息的相同信息。OSPF 和 IS-IS 即使在同一网络上运行，也不会交互，除非它们被配置为交互（例如通过重新分配）。OSPF 和 BGP 交互方式相同吗？否，因为 BGP 依赖于底层 IGP 来提供用于构建对等关系和下一跳信息的 IP 可达性。使用此处描述的模型，OSPF 和 BGP 之间的交互面比 OSPF 和 IS-IS 之间的交互面更大，

These concepts will be related to network design and architecture more fully as this book works through various examples, but consider this: OSPF and Intermediate System-to-Intermediate System (IS-IS), for instance, run as “ships in the night” routing protocols. They both rely on the same information about IP addresses, link states, metrics, and other information drawn directly from the network topology. OSPF and IS-IS, even if they’re running on the same network, don’t interact unless they are configured to interact (through redistribution, for instance). Do OSPF and BGP interact in the same way? No, because BGP relies on the underlying IGP to provide IP reachability for building peering relationships and next hop information. Using the model described here, the interaction surface is larger between OSPF and BGP than it is between OSPF and IS-IS, and the APIs are fairly opaque.

状态多于实现目标所需的任何事物

Anything for Which There Is More State Than Required to Achieve a Goal

几乎工程界的每个人都看过鲁布·戈德堡机器的漫画，如图1.3所示。⁵

Virtually anyone in the engineering world has seen a cartoon of a Rube Goldberg Machine, such as the one in Figure 1.3.⁵

5 . 公共区域; 摘自：“Professor_Lucifer_Butts.gif (428 × 302)”，np，2014 年 7 月 7 日访问，https://upload.wikimedia.org/wikipedia/commons/a/a6/Professor_Lucifer_Butts.gif。

5. Public domain; taken from: “Professor_Lucifer_Butts.gif (428 × 302),” n.p., accessed July 7, 2014, https://upload.wikimedia.org/wikipedia/commons/a/a6/Professor_Lucifer_Butts.gif.

图 1.3 Rube Goldberg 机器

Figure 1.3 A Rube Goldberg Machine

这些装置常常被贴上解决某些问题的简单方法的标签，从吃饭时用餐巾（如图1.3所示）到顶起汽车更换轮胎（诱导汽车顶上的大象移动到平台上）将汽车的一端撬到空中）。简单是关键词，因为这些机器显然一点也不简单。但为什么这样的机器被认为很复杂——甚至复杂到了幽默的地步？因为它们说明了有关复杂性本质的两个具体观点。

Quite often these contraptions were labeled as a simple way to solve some problem, from using a napkin during a meal (as in Figure 1.3) to jacking a car up to replace the tire (an elephant atop the car is induced to move onto a platform that levers one end of the car up into the air). Simple is the key word, because these machines were obviously anything other than simple. But why are such machines considered complex—even to the point of being humorously complex? Because they illustrate two specific points about the nature of complexity.

鲁布·戈德堡的装置总是针对简单问题的多步骤解决方案。全自动餐巾机如图1.3所示例如，取代了拿起餐巾纸擦嘴的简单动作。当你显然有一只手可以用来完成同样的任务时，使用这样的机器是荒谬的——即使你的另一只手没有空，把叉子放在桌子上然后拿起餐巾比用它来完成任务更简单。建造这台疯狂的机器来做同样的事情。那么，您对复杂性的看法与正在解决的问题和所提供的解决方案之间的关系有关。任何添加不必要的步骤、交互或部分来解决特定问题的系统都被视为复杂的，无论问题或解决方案客观地来看多么简单。

Rube Goldberg’s contraptions are always multistep solutions for a simple problem. The automatic napkin machine shown in Figure 1.3, for instance, replaces the simple action of picking the napkin up and wiping your mouth with it. It’s absurd to use such a machine when you obviously have one hand free that could be used for the same task—and even if your other hand isn’t free, it’s simpler to put a fork down on the table and pick up a napkin than to build this crazy machine to do the same thing. Your perception of complexity, then, is related to the relationship between the problem being solved and the solution offered. Any system that adds unnecessary steps, interactions, or parts to solve a specific problem is seen as complex, no matter how simple the problem or the solution when viewed objectively.

鲁布·戈德堡的装置总是专注于手头的问题，而排除所提出的解决方案的副作用。自动餐巾需要每次使用时必须更换的火箭、必须喂食的小鸟、必须上弦和维护的时钟以及每次使用后必须更换的饼干。这个解决方案真的比拿起餐巾纸更简单吗？用于撬起汽车以便更换轮胎的大象必须被搬运和喂食。当大象睡着了，需要更换轮胎时，车主该怎么办？

Rube Goldberg’s contraptions always focus on the problem at hand to the exclusion of the side effects of the solution proposed. The automatic napkin requires a rocket that must be replaced each time it is used, a bird that must be fed, a clock that must be wound and maintained, and a biscuit that must be replaced after each use. Is the solution really simpler than picking a napkin up? The elephant used to lever the car up so the tire can be replaced must be carried and fed. What should the owner of the car do when the elephant is asleep and the tire needs to be replaced?

鲁布·戈德堡的喜剧天才和他的神奇机器告诉我们：

The comic genius of Rube Goldberg and his fabulous machines teaches us:

• 解决方案的复杂性必须与要解决的问题直接相关。如果有更简单的解决方案可用，工程师将（并且应该）倾向于该解决方案。将此称为工程复杂性的“奥卡姆剃刀”：如果提出了两种解决方案来解决问题，则应始终首选较简单的解决方案，因为两种解决方案都能同样出色地解决问题。

• The complexity of the solution must be directly related to the problem being solved. If there is a simpler solution available, engineers will (and should) gravitate to that solution. Call this the “Occam’s Razor” of complexity in engineering: if two solutions have been proposed to resolve a problem, the simpler solution should always be preferred, given both solutions solve the problem equally well.

• 如果焦点过于集中，很容易造成不必要的复杂性。一个更简单的解决方案实际上可能看起来更简单，直到它满足未来的考验或与现实结合。设计一个仅包含所需编码的协议似乎更容易，并且没有像 TLV 这样混乱的结构，至少在您需要修改协议来解决某些问题之前是这样。您在进行初始设计工作时没有想到的问题。换句话说，（显然）最干净、最简单的设计往往会出现意想不到的脆弱性。同样，在实际部署解决方案之前，并非特定解决方案的所有问题都是显而易见的。

• It is easy to create unnecessary complexity by narrowing the focus too far. A simpler solution might actually seem simpler until it meets the test of the future, or engages with reality. It might seem easier to design a protocol with just the encodings needed, and without messy constructions like TLVs—at least until you need to modify the protocol to address some problem you didn’t think of when doing the initial design work. To put this another way, there is often an unexpected brittleness to the (apparently) cleanest and simplest design available. In the same way, not all the problems with a particular solution are obvious until the solution is actually deployed.

当您考虑网络架构和网络协议设计的现实世界中复杂性的具体实例时，这两个原则的示例会定期出现。

Examples of these two principles will crop up on a regular basis as you consider specific instances of complexity in the real worlds of network architecture and network protocol design.

意想不到的后果

Unintended Consequences

1996 年，讨论世界上最著名的足球比赛超级碗的网站被各种搜索引擎屏蔽。为什么互联网上的搜索引擎会屏蔽与足球相关的网站？因为每届超级碗都是从 1967 年第一场比赛 Super Bowl I 开始按顺序编号的。名称中的数字始终采用罗马数字。1996 年超级碗是第 30 场比赛，因此被恰当地称为 Super Bowl XXX。但 XXX 也代表了互联网上广泛使用的一种特定类型的材料，许多人不希望在搜索结果中看到这种材料。因此，超级碗XXX因为“XXX”而被许多搜索引擎屏蔽。⁶

In 1996, websites discussing the Super Bowl, the most famous football game in the world, were being blocked by various search engines. Why would any search engine on the Internet block sites related to football? Because each Super Bowl is numbered sequentially starting in 1967 with the first game, Super Bowl I. The numbers in the name have always been in roman numerals. The 1996 Super Bowl was game number 30, so it was appropriately called Super Bowl XXX. But XXX also represents a particular type of material widely available on the Internet that many people don’t want to see in their search results. Hence, Super Bowl XXX, because of the “XXX,” was blocked from many search engines.⁶

6 . “电子费率和过滤：儿童互联网保护法审查”（一般。能源和商业，电信和互联网小组委员会，2001 年 4 月 4 日），np，2014 年 7 月 8 日访问，http://www . gpo.gov/fdsys/pkg/CHRG-107hhrg72836/pdf/CHRG-107hhrg72836.pdf。

6. “E-Rate and Filtering: A Review of the Children’s Internet Protection Act” (General. Energy and Commerce, Subcommittee on Telecommunications and the Internet, April 4, 2001), n.p., accessed July 8, 2014, http://www.gpo.gov/fdsys/pkg/CHRG-107hhrg72836/pdf/CHRG-107hhrg72836.pdf.

这个（也许很有趣）的故事是试图审查互联网内容的历史上一长串此类事件之一，但它更强调了意外后果的力量。1996 年，罗伯特·K·默顿 (Robert K. Merton) 列出了意外后果的五个来源。⁷其中三个直接适用于大型计算机网络系统：

This (perhaps amusing) story is one of a long chain of such incidents in the history of attempting to censor content on the Internet—but it makes a larger point about the power of unintended consequences. In 1996, Robert K. Merton listed five sources of unintended consequences.⁷ Three of these directly apply to large computer network systems:

7 . 罗伯特·K·默顿，“有目的的社会行动的意外后果”，《美国社会学评论》 1，第 1 期。6（1936 年 12 月 1 日）：894–904，2015 年 9 月 15 日访问，http://www.jstor.org/stable/2084615。

7. Robert K. Merton, “The Unanticipated Consequences of Purposive Social Action,” American Sociological Review 1, no. 6 (December 1, 1936): 894–904, accessed September 15, 2015, http://www.jstor.org/stable/2084615.

• 无知，无法预测一切，从而导致分析不完整

• Ignorance, making it impossible to anticipate everything, thereby leading to incomplete analysis

• 分析问题时出现错误，或遵循过去有效但可能不适用于当前情况的习惯

• Errors in analysis of the problem or following habits that worked in the past but may not apply to the current situation

• 眼前利益高于长远利益

• Immediate interests overriding long-term interests

意外后果的概念如何应用于网络的复杂性？随着系统变得更加复杂，根据任何给定的输入来预测输出变得更加困难。例如，对互联网路由系统中各种输入的真实结果进行了大量研究——拓扑更新或可达性在整个互联网上传播需要多长时间，一个提供商如何改变或实施特定政策会影响其他提供商，以及看似简单的想法（例如抑制路线抖动）如何在现实世界中发挥作用？必须进行研究才能找到这些问题的答案，这意味着互联网总体上是一个复杂的系统。给定特定的输入，您可以猜测可能的结果，

How does the concept of unintended consequences apply to the complexity of a network? As a system becomes more complex, it becomes more difficult to predict the output based on any given input. As an example, a lot of research is done into the real results of various inputs into the Internet routing system—how long does it take for an update in topology or reachability to propagate throughout the Internet as a whole, how does one provider changing or implementing a specific policy impact other providers, and how do seemingly straightforward ideas, such as route flap dampening work in the real world? That research must be undertaken to discover the answers to these questions implies that the Internet, at large, is a complex system. Given a specific input, you can guess at a likely outcome, but there’s no assurance that the outcome will be what you expect.

具有大量互连部件的大型复杂系统很难分析，导致分析不完整，从而因无知而产生意想不到的后果。在一个足够大的系统中，有足够多的组件具有透明的交互和大的交互表面，几乎不可能知道单个更改将影响系统的所有方式。

Large complex systems with a lot of interconnected parts are difficult to analyze, leading to incomplete analysis, and hence unintended consequences through ignorance. In a large enough system, with enough components that have transparent interactions and large interaction surfaces, it’s almost impossible to know all the ways in which a single change will affect the system.

同样，大型复杂系统往往更多地通过经验（凭感觉飞翔）和经验法则来管理——如果遵循规则，进行全面分析的成本通常被认为比失败的成本高得多。大拇指的方向是错误的。这再次涉及到规模和子系统相互作用的复杂性问题。系统越复杂，分析中的错误就越有可能渗透到日常运营模型中。

In the same way, large complex systems tend to be managed more by experience (seat of the pants flying) and rule of thumb—the cost to do a full analysis is often perceived to be much higher than the cost of a failure if the rule of thumb is wrong. This again relates to the problem of complexity through the scale and subsystem interaction. The more complex a system, the more likely errors in analysis are to creep into everyday operational models.

笔记

Note

在飞行员拥有可以告诉他们飞机角度（称为偏航、俯仰和滚转）的仪器之前，他们会“凭感觉飞行”。从字面上看，这意味着他们会根据转弯时是否在座位上滑动来判断飞机的速度与转弯的锐度。如果你在座位上滑动，那么你的飞行速度不够快，无法转弯——转弯时飞机速度的离心力应该使飞行员保持在座位上。

Before pilots had instruments that could tell them the angle of the airplane (called yaw, pitch, and roll), they would “fly by the seat of their pants.” This literally means that they would judge the speed of the plane in relation to the sharpness of a turn by whether or not they slid in their seat when making the turn. If you were sliding in your seat, you weren’t flying fast enough for the turn—the centrifugal force of the speed of the plane in the turn should keep the pilot in place in the seat.

最后一个问题是每个网络工程师都熟悉的问题——凌晨两点的电话、应用程序宕机，如果不“立即启动”，公司将损失数百万美元，以及获取信息的捷径回来工作。我们总是告诉自己，我们会在早上查看它，或者我们会将其放在待办事项列表中，以便稍后修复，但是立即的暴政很快就会接管，并且黑客仍然作为正常运行情况网络的。一个由一千个路由器组成的网络中的一次黑客攻击似乎不会产生很多负面后果。但实际上，一次黑客攻击就可以迅速瘫痪一千台路由器的网络，而在拥有一万台路由器的网络中遭受一千次黑客攻击，一场灾难即将发生。

The final problem is one every network engineer knows well—it’s the two-in-the-morning phone call, the application that’s down and will cost the company millions if it’s not “up—right now,” and the shortcut taken to get things back working. We always tell ourselves we’ll look at it in the morning, or we’ll put it on the to-do list to fix sometime later, but the tyranny of the immediate takes over soon enough, and the hack stays in as part of the normal operational profile of the network. A single hack in a network of a thousand routers might not seem like it will have many negative consequences. In reality, however, a single hack can quickly bring down a thousand-router network, and a thousand hacks in a network of ten thousand routers, however, are a disaster just waiting to happen.

笔记

Note

仅仅为了解决当前问题而不考虑未来影响而构建复杂性概念的另一个术语是技术债务。⁸

Another term for the concept of building in complexity simply to address a problem at hand, without considering the future impact, is technical debt.⁸

8 . “技术债务”，维基百科，免费百科全书，2015 年 9 月 2 日，2015 年 9 月 15 日访问，https://en.wikipedia.org/w/index.php ?title=Technical_debt&oldid= 679133748。

8. “Technical Debt,” Wikipedia, the Free Encyclopedia, September 2, 2015, accessed September 15, 2015, https://en.wikipedia.org/w/index.php?title=Technical_debt&oldid=679133748.

意外后果的力量告诉我们，为了更好地理解和管理网络复杂性，工程师需要专注于分析和理解用于构建功能网络的各种组件之间的各种状态和交互的能力。您可以应用越多的工具和概念来理解网络可能陷入的各种状态（例如模型和测量工具），您就越能够更好地处理现实世界中的复杂性。同时，人类的理解是有限的，因此任何人可以预见的副作用的数量也是有限的。

The power of unintended consequences teaches that to better understand, and manage, network complexity, engineers need to focus on the ability to analyze and understand the various states and interactions between the various components used to build a functioning network. The more tools and concepts you can apply to understanding the various states into which a network can fall—such as models and measurement tools—the better you will be able to deal with complexity in the real world. At the same time, there is a limit to human understanding, and therefore a limit to the number of side effects anyone can foresee.

为什么这么复杂？

Why So Much Complexity?

如果复杂性如此——复杂——那么为什么不设计更简单的网络和协议呢？换句话说，为什么从长远来看，网络世界中每一次让事情变得更简单的尝试最终都会使事情变得更加复杂？例如，通过在 IP 之上（或通过 IP）建立隧道，可以降低控制平面的复杂性，并且整体上使网络变得更简单。那么，为什么隧道覆盖层最终会包含如此多的复杂性呢？

If complexity is so—complex—then why not just design networks and protocols that are simpler? To put the question another way, why does every attempt to make anything simpler in the networking world end up apparently making things more complex in the long run? For instance, by tunneling on top of (or through) IP, the control plane’s complexity is reduced, and the network is made simpler overall. Why is it, then, that tunneled overlays end up containing so much complexity?

这个问题有两个答案。首先，人性就是如此，工程师总是会发明十种不同的方法来解决同一问题。在虚拟世界中尤其如此，新的解决方案（相对）容易部署，（相对）容易发现最后一组提出的解决方案的问题，并且（相对）容易移动一些位来创建新的解决方案。一种“比旧解决方案更好”的新解决方案。换句话说，虚拟空间之所以如此混乱，是因为在那里很容易构建新的东西。

There are two answers to this question. The first is that human nature being what it is, engineers will always invent ten different ways to solve the same problem. This is especially true in the virtual world, where new solutions are (relatively) easy to deploy, it’s (relatively) easy to find a problem with the last set of proposed solutions, and it’s (relatively) easy to move some bits around to create a new solution that is “better than the old one.” The virtual space, in other words, is partially so messy because it’s so easy to build something new there.

然而，第二个答案在于一个更根本的问题：复杂性对于处理难以解决的问题所涉及的不确定性是必要的。奥尔德森和多伊尔指出：

The second answer, however, lies in a more fundamental problem: complexity is necessary to deal with the uncertainty involved in difficult to solve problems. Alderson and Doyle state:

然而，我们认为，最简洁地讨论复杂性是在功能及其稳健性方面。具体来说，我们认为，高度组织化的系统的复杂性主要源于旨在为其环境和组件的不确定性创造稳健性的设计策略。⁹

In our view, however, complexity is most succinctly discussed in terms of functionality and its robustness. Specifically, we argue that complexity in highly organized systems arises primarily from design strategies intended to create robustness to uncertainty in their environments and component parts.⁹

9 . David L. Alderson 和 John C. Doyle，“复杂性的对比观点及其对以网络为中心的基础设施的影响”，IEEE Transactions on Systems, Man, and Cybernetics 40，no。4（2010 年 7 月）：840。

9. David L. Alderson and John C. Doyle, “Contrasting Views of Complexity and Their Implications for Network-Centric Infrastructures,” IEEE Transactions on Systems, Man, and Cybernetics 40, no. 4 (July 2010): 840.

这种说法可以用图1.4所示的图表来表达。

This statement can be expressed in a chart as shown in Figure 1.4.

图 1.4 复杂性、稳健性和解决方案有效性

Figure 1.4 Complexity, Robustness, and Solution Effectiveness

这是违反直觉的——事实上，它几乎与大多数有关网络工程的讨论相反。工程师通常认为增加简单性会增加稳健性，但事实并非如此。相反，增加复杂性会增加鲁棒性，直到解决方案超出鲁棒性曲线上的峰值。为什么会这样呢？因为不确定性。作为一个简单的例子，让我们回到网络协议中经常使用的 TLV 编码。哪个更好？

This is counterintuitive—in fact, it’s almost the opposite of most discussions around network engineering. Engineers often assume that increasing simplicity leads to increasing robustness—but this is not true. Instead, increasing complexity increases robustness until the solution moves beyond the peak on the robustness curve. Why should this be? Because of uncertainty. As a simple example, let’s return to TLV encodings often used in network protocols. Which is better?

• 设计一个能够以其原始格式处理大量情况的协议，并且还支持许多在设计协议时没有想到的不同扩展。

• Designing a protocol that can handle a large number of situations in its original format, and also support many different extensions that hadn’t been thought of when the protocol was designed.

• 设计一个协议，使用尽可能少的信息集来支持设计阶段一开始提出的要求。

• Designing a protocol that will support, using the minimal set of information possible, the requirements laid out at the very beginning of the design phase.

在单一协议的情况下，对于第二种选择有一个强有力的论据——设计协议以支持设计阶段开始时提出的要求，并使用所需的最少信息。具体原因有两个；第二个可能看起来是最优的。

There is a strong argument to be made, in the single protocol case, for the second option—designing the protocol to support the requirements presented at the beginning of the design phase with the minimal amount of information required. There are two specific reasons; the second might appear to be the most optimal.

优化设计的协议的在线配置文件始终小于采用灵活附加功能设计的协议。对于任何 TLV，都必须有一个 TLV 标头 — 必须描述值的类型和长度。另一方面，任何旨在以最佳方式仅携带一组特定信息的协议都不需要携带任何有关所携带信息是什么的信息。换句话说，如果协议要足够灵活以在将来添加新的数据类型，则必须随数据携带元数据或数据描述。然而，如果协议是“封闭的”，则元数据是协议描述的一部分，并且不需要与数据一起在线路上传送。灵活协议中的元数据是内部化的，或者与数据本身保持一致，以创造灵活性；它被外部化，或者位于协议实现中，以优化线路上的位的使用。

The on-the-wire profile of an optimally designed protocol will always be smaller than one designed with flexible additions. For any TLV, there must be a TLV header—something must describe the type and length of the value. On the other hand, any protocol that is designed to optimally carry just a specific set of information doesn’t need to carry any information about what the carried information is. To put this in other terms, the metadata, or data description, must be carried with the data if the protocol is to be flexible enough to add new data types in the future. If the protocol is “closed,” however, the metadata is part of the protocol description, and need not be carried on the wire with the data. The metadata in a flexible protocol is internalized, or carried in line with the data itself to create flexibility; it is externalized, or located in the protocol implementation, to create optimal use of bits on the wire.

优化设计的协议的处理配置文件总是比通过灵活添加而设计的协议更好。同样，围绕 TLV 或未来允许承载更多类型信息的任何其他格式设计的协议将需要更复杂的处理。数据包中的偏移量不能用于查找数据包中任何位置包含的任何信息 — 相反，必须“遍历”数据流以查找下一个 TLV 标头，并且必须根据每个 TLV 的一组 TLV 进行处理规则。有关此概念的说明请参见图1.5 。

The processing profile of an optimally designed protocol will always be better than one designed with flexible additions. In the same way, a protocol designed around TLVs, or any other format that allows more types of information to be carried in the future, will require more complex processing. Offsets into the packet cannot be used to find any piece of information contained anywhere in the packet—instead, the data stream must be “walked,” to find the next TLV header, and the TLV must be processed according to a set of per TLV rules. See Figure 1.5 for an illustration of this concept.

图 1.5 TLV 与最佳结构化数据包格式

Figure 1.5 TLV versus Optimally Structured Packet Formats

比较两种情况下查找包含 X 值的八位字节所需的处理。对于最佳格式的数据包：

Compare the processing required to find the octet containing the value of X in both cases. For the optimally formatted packet:

• 数出14 个八位字节。

• Count off 14 octets.

• 从第14 个八位位组的内容中读取X 的值。

• Read the value of X from the contents of the 14th octet.

对于TLV格式的数据包：

For the TLV formatted packet:

• 读取第一个TLV 标头。

• Read the first TLV header.

• 这是Y TLV；找到长度并跳到数据包中 TLV 的末尾。

• This is a Y TLV; find the length and skip to the end of the TLV in the packet.

• 读取第二个TLV 标头。

• Read the second TLV header.

• 这是一个Z TLV，找到长度并跳到数据包中TLV 的末尾。

• This is a Z TLV, find the length and skip to the end of the TLV in the packet.

• 读取第三个TLV 标头。

• Read the third TLV header.

• 这是X TLV。

• This is an X TLV.

• 根据该特定TLV 的格式跳转到X TLV，并读取X 的值。

• Jump into the X TLV, based on the format of this particular TLV, and read the value of X.

对于最佳格式的数据包的处理要简单得多；处理 TLV 需要将更多位移入内存或移出内存、进行检查等。针对承载非常具体的数据片段而优化的协议可以组织该数据，使处理更容易，从而降低处理器利用率。这在数据包交换领域尤其重要，在数据包交换领域，定制硬件用于以非常高的速率交换数据包，以及其他使用硬件近实时处理数据包的地方。

The processing for the optimally formatted packet is much simpler; processing TLVs requires more bits to be moved into and out of memory, examined, etc. Protocols optimized to carry very specific pieces of data can have that data organized to make processing easier, and hence to reduce processor utilization. This is particularly important in the area of packet switching, where customized hardware is used to switch packets at a very high rate, and other places where hardware is used to process packets in near real time.

然而，在这个等式的另一边，存在意想不到的（或不可预测的）情况。使用 TLV 示例可以让您以两种方式了解这一点：未来的扩展和错误处理。

On the other side of this equation, however, there is the unexpected (or unpredicted). Using the TLV example allows you to see this in two ways: future extensions and error handling.

未来的扩展与新协议

Future Extensions versus New Protocols

假设您设计了一些新协议，该协议在线路和处理要求方面都进行了完美优化，以传输有关一家大公司每个工厂每天生产的小部件数量的信息。很快，该公司看到了为任何想要购买这些小部件的人提供贷款的机会，并且出现了一项新要求：通过网络传输小部件贷款信息的能力。本着完美优化网络性能的精神，您设计了一种新协议来通过网络传输贷款信息 - 同样，该协议旨在最大限度地减少整个网络的带宽利用率和处理要求。随着贷款业务的扩大，该公司认为高金融是一项不错的业务，因此他们决定扩大其网点，不仅销售小部件，还销售五到六种其他商品，并为每种商品提供融资。问题很快就变成了 - 您是否应该继续设计和部署单独的协议来单独管理每个新需求，或者您是否应该首先设计一个可以管理更广泛需求的单一、灵活的协议？

Assume you’ve designed some new protocol that is perfectly optimized both on the wire and in terms of processing requirements to transport information about the number of widgets being produced on a daily basis in each factory at a large company. Soon enough, the company sees an opportunity in offering loans for anyone who wants to buy one of these widgets, and a new requirement arises: the ability to transport information about loans for widgets across the network. In the spirit of perfectly optimized network performance, you design a new protocol to transport loan information across the network—again, the protocol is designed to minimize bandwidth utilization and processing requirements throughout the network. As the loan business expands, the company decides high finance is a good business to be in, so they decide to expand their outlets to sell not just widgets, but five or six other items, and to provide financing for each of those items as well. The question quickly becomes—should you continue designing and deploying individual protocols to manage each new requirement separately, or should you have designed a single, flexible protocol that could manage a wider range of requirements in the first place?

在前面讨论的 TLV 格式示例中，TLV 格式的数据包需要更多的在线带宽和更多的处理能力，但它们也允许使用单个协议来计算小部件、小部件的贷款、一般贷款和其他产品一般。如果协议被设计为在一组目标内管理广泛的数据类型，那么系统实际上最终会比每个目标通过单独的协议来满足的系统更简单。

In the TLV format example discussed previously, TLV formatted packets require more on-the-wire bandwidth, and more power to process, but they also allow for a single protocol to be used for counting widgets, loans for widgets, loans in general, and other products in general. If a protocol is designed to manage a broad range of data types within a single set of goals, then the system actually ends up being simpler than one in which each goal is met with a separate protocol.

这个例子可能有点牵强，但当你进入本书更实用的部分时，你会变得更加明显，这是设计师和建筑师经常面临的一个真正的问题。总是存在一种通过简单地“在顶部”放置一个新协议来扩展网络的诱惑，但如果没有关于目标、功能分离和域分离的一些扎实的想法，“在顶部”很快就会成为“意大利面条”的委婉说法。盘子的顶部。” 这种现象的另一个例子是将专为某一目的设计的协议置于完全不同的角色。例如，资源预留协议 (RSVP) 最初设计用于为特定数据包流预留沿路径的队列和处理空间。

This example might be a little stretched, but as you get into the more practical sections of this book it will become ever more apparent just how it is a real question designers and architects face on a regular basis. There is always a temptation to extend the network by simply putting a new protocol “over the top,” but without some solid ideas about goals, functional separation, and domain separation, “over the top,” quickly becomes a euphemism for “spaghetti on top of the plate.” Another example of this phenomenon is putting a protocol designed for one purpose into a completely different role. For instance, the Resource Reservation Protocol (RSVP) was originally designed to reserve queue and processing space along a path for a particular flow of packets. The most common use for RSVP today is the signaling of traffic engineering paths through a network—a purpose far outside the original design of the protocol.

意外错误

Unexpected Errors

现实世界中不确定性的另一个来源是错误。无论出于何种原因，事情并不总是按计划进行。协议或网络如何对这些意外事件做出反应是一个至关重要的考虑因素，尤其是当网络成为生活“正常”的一部分，从游戏到金融交易再到医疗程序的一切都依赖网络时。

Another source of uncertainty in the real world is errors; for whatever reason, things don’t always go as planned. How a protocol or network reacts to these unexpected events is a crucial consideration, especially as networks become a “normal” part of life, relied on for everything from gaming to financial transactions to medical procedures.

在处理网络错误方面，为了增加鲁棒性而增加复杂性的一个例子是许多协议中的错误检测或纠正代码。图 1.6说明了一个简单的基于奇偶校验的方案。

An example of added complexity for added robustness in the area of handling network errors is the error detection or correction code found in many protocols. Figure 1.6 illustrates a simple parity-based scheme.

图 1.6 奇偶校验位和错误检测

Figure 1.6 The Parity Bit and Error Detection

奇偶校验位是错误检测码的一个简单示例。建包时，会统计二进制 1 的总数，以确定 1 的个数是偶数还是奇数。如果1的数量是奇数，则将奇偶校验位设置为1，使1的数量为偶数。如果接收到的数据包中 1 的总数（包括奇偶校验位）为奇数，则接收方知道该数据包在传输过程中一定已以某种方式损坏。这不会告诉接收者正确的信息是什么，但它确实让接收者知道它需要请求数据的另一个副本。

The parity bit is a simple example of an error detection code. When the packet is built, the total number of binary 1s is counted to determine if there is an even or odd number of 1s. If the number of 1s is odd, then the parity bit is set to 1 to make the number of 1s even. If a packet is received where the total number of 1s, including the parity bit, is odd, the receiver knows that the packet must have been corrupted during transmission in some way. This doesn’t tell the receiver what the correct information is, but it does let the receiver know it needs to ask for another copy of the data.

添加奇偶校验位会增加数据包处理的复杂性。每个数据包必须存储在某个位置的缓冲区中，同时计算数据包中 1 的数量，忽略奇偶校验位本身。如果 1 的数量是奇数，则必须在数据包传输之前设置奇偶校验位。同样，接收方在接受数据之前必须采取额外的步骤来计算数据包中 1 的数量，并且协议必须内置某种机制，以便接收方请求另一份信息副本。这种增加的复杂性值得吗？这完全取决于在正常操作中通过此类系统检测到的故障发生的频率，或者单个故障的灾难性程度。随着错误检测或纠正机制的复杂性增加，应用程序从格式错误的传输中恢复的能力也得到了增强。代价是额外的数据包处理，以及通过添加纠错码本身可能引入的错误。

Adding a parity bit increases the complexity of packet processing. Each packet must be stored in a buffer someplace while the number of 1s in the packet are counted, ignoring the parity bit itself. If the number of 1s is odd, then the parity bit must be set before the packet is transmitted. The receiver must likewise take the extra step of counting the number of 1s in the packet before accepting the data, and the protocol must have some mechanism built in for the receiver to ask for another copy of the information. Is this added complexity worth it? It all depends on how often failures that can be detected through such a system happen in normal operation, or how catastrophic a single failure would be. As the complexity of the error detection or correction mechanism ramps up, the ability of the application to recover from malformed transmissions is also increased. The cost is additional packet processing, along with the errors potentially introduced through the addition of the error correction code itself.

为什么不构建无限复杂的系统？

Why Not Build Infinitely Complex Systems?

增加复杂性可以使网络更轻松地处理未来的需求和意外事件，并通过较小的基本功能集提供更多服务。如果是这种情况，为什么不简单地构建一个在单个网络上运行的单个协议，该协议可以处理可能抛出的所有需求，并且可以处理您可以想象的任何事件序列？运行单一协议的单一网络肯定会减少网络工程师需要处理的移动部件的数量，使我们的生活变得更简单，对吧？

Adding complexity, then, allows a network to handle future requirements and unexpected events more easily, as well as provides more services over a smaller set of base functions. If this is the case, why not simply build a single protocol running on a single network that can handle all the requirements potentially thrown at it, and can handle any sequence of events you can imagine? A single network running a single protocol would certainly reduce the number of moving parts network engineers need to deal with, making all our lives simpler, right?

也许不会。在某些时候，任何复杂的系统都会变得脆弱——强大但脆弱是一个可以用来描述这种情况的短语。当一个系统能够对一组预期的情况做出弹性反应时，它是强大但脆弱的，但一组意外的情况将导致它失败。举一个现实世界的例子——刀片需要具有某种独特的特性组合。它们必须足够坚硬，可以固定边缘并进行切割，但又必须足够灵活，可以在使用时稍微弯曲，恢复到原来的形状，没有任何损坏的迹象，并且在跌落时不得破碎。需要多年的研究和经验才能找到合适的金属来制造刀片，并且仍然存在关于哪种材料适合特定性能、在什么条件下等的长期而深入的技术讨论。

Maybe not. At some point, any complex system becomes brittle—robust yet fragile is one phrase you can use to describe this condition. A system is robust yet fragile when it is able to react resiliently to an expected set of circumstances, but an unexpected set of circumstances will cause it to fail. To give an example from the real world—knife blades are required to have a somewhat unique combination of characteristics. They must be hard enough to hold an edge and cut, and yet flexible enough to bend slightly in use, returning to their original shape without any evidence of damage, and they must not shatter when dropped. It has taken years of research and experience to find the right metal to make a knife blade from, and there are still long and deeply technical discussions about which material is right for specific properties, under what conditions, etc.

为了理解复杂性，制作刀片时有一个特别令人感兴趣的阶段：回火过程。为了回火刀片，首先将刀片加热到非常高的温度，然后冷却。重复此过程数次可使钢的分子排列，从而在钢内形成晶粒，如图1.7所示。¹⁰

There is one specific stage of making a knife blade of particular interest in the quest to understand complexity: the tempering process. To temper a knife blade, the blade is first heated to a very high temperature, and then allowed to cool. Repeating this process several times aligns the molecules of the steel so it forms a grain within the steel, as shown in Figure 1.7.¹⁰

10 . 图片取自http://practicalmaintenance.net/wp-content/uploads/High-carbon-AISI-1095-Steel.jpg。

10. Image taken from http://practicalmaintenance.net/wp-content/uploads/High-carbon-AISI-1095-Steel.jpg.

图 1.7 淬硬钢中的晶粒

Figure 1.7 Grain in Hardened Steel

这种纹理就像木材中的纹理一样，产生尺寸强度，从而使所得的钢材非常坚硬——事实上，通过这个过程，钢材会变得非常坚硬，如果掉落在坚硬的表面（例如混凝土）上，就会破碎。或瓷砖地板。此时的刀片坚固但脆弱。它能够很好地实现了其主要设计（切割材料），但它对意外事件（掉落在坚硬表面上或施加扭转）的反应不佳。为了使刀可用，刀片在实际使用之前必须进行“回火”。为此，将钢加热（通常加热到比钢硬化时使用的温度稍低的温度），然后在油浴中淬火。这一过程破坏了硬化过程中产生的晶粒的稳定性，使钢材稍微变软，但在此过程中，钢材也变得更加柔韧。该过程的最终结果是刀片具有锋利、足够坚硬以进行切割，但又足够灵活以适合日常使用。

This graining acts just like the grain in wood, creating dimensional strength, thus making the resulting piece of steel very hard—in fact, steel can become so hard through this process that it can shatter if dropped on a hard surface, such as a concrete or tile floor. The blade, at this point, is robust yet fragile; it is able to achieve its primary design (cutting material) very well, but it doesn’t react well to unexpected events (being dropped on a hard surface, or having torsion applied). To make the knife useable, the blade must be “detempered,” before it is actually used. To do this, the steel is heated (normally to a slightly lower temperature than used in hardening the steel), and then quenched in a bath of oil. This process destabilizes the graining the hardening process has created, making the steel slightly softer—but in the process, the steel also becomes much more flexible. The end result of the process is a blade that holds an edge, is hard enough to cut, and yet is flexible enough for everyday use.

通过强化过程创建的粒度代表了以各种方式放入网络和协议中的复杂性，例如通过添加 TLV 将元数据添加到数据包格式中，或者在源和目标之间添加更多路径，甚至自动化通常处理的过程人类在处理网络更改时所犯的错误数量。还需要以分层协议的形式进行扰乱过程，为网络的不同部分设置一般（通常是狭窄的）目标，打破故障域等。

The grain created through the hardening process represents the complexity put into networks and protocols in various ways, such as adding metadata into the packet format by adding TLVs, or adding more paths between the source and destination, or even automating a process that is normally handled by humans to reduce the number of mistakes made in handling changes to the network. The distempering process is also needed, as well, in the form of layering protocols, setting general (and often narrow) goals for different parts of the network, breaking up failure domains, etc.

那么，复杂性可以被视为一种权衡。如果在一个方向上降低规模太远，您最终会得到一个没有弹性的网络，因为没有冗余，或者存在单个故障域等。如果在另一个方向上降低规模，您最终会得到一个没有弹性的网络，因为协议和系统无法应对快速变化。在这个规模上不存在“完美点”——就像钢铁一样，这完全取决于网络工程师要实现的目标。另外两个来自技术领域的插图将有助于巩固这一概念。

Complexity, then, can be seen as a tradeoff. If you go too far down the scale in one direction, you wind up with a network that isn’t resilient because there is no redundancy, or there is a single failure domain, etc. If you go down the scale in the other direction, you wind up with a network that isn’t resilient because the protocols and systems cannot cope with a rapid change. There is no “perfect point” on this scale—just as with steel, it all depends on the goals the network engineer sets out to meet. Two more illustrations, taken from the technical world, will help to cement this concept.

快速、便宜、高质量：二选一

Quick, Cheap, and High Quality: Choose Two

几乎每个人都知道这一点，但值得定期重复。面对任何决定，您都会有三个目标：快速、便宜和高质量。在这三个中，您可以选择任意两个，但不能选择全部三个。如果您选择廉价且快速的解决方案，那么您几乎肯定不会最终获得高质量的解决方案。如果选择质优价廉的解决方案，实施起来会需要很长时间。您可以用一种不寻常的方式可视化这种三向权衡，如图1.8所示。

Just about everyone knows this, but it bears repeating on a regular basis. Faced with just about any decision, you will have three goals: quick, cheap, and high quality. Of those three, you can choose any two—but never all three. If you choose a solution that is cheap and quick, you will almost certainly not end up with a quality solution. If you choose a solution that is high quality and cheap, it will take a long time to implement. You can visualize this three-way tradeoff in a somewhat unusual way, as shown in Figure 1.8.

图 1.8 快速、廉价、质量难题

Figure 1.8 The Quick, Cheap, Quality Conundrum

在图 1.8中，较暗的阴影区域可以称为“现实领域”。更大、阴影更浅的三角形可以称为“目标领域”。虽然目标包含所有三种可能性，但现实是结构化的，因此您只能实现给定目标的一部分。您可以选择（如最左侧）平均平衡所有三个目标。在下一个插图中，焦点已经转向快速，从而导致远离廉价和高质量。

In Figure 1.8, the darker shaded area is what might be called “the realm of reality.” The larger, more lightly shaded triangle is what might be called “the realm of goals.” While the goals contain all three possibilities, reality is structured so you can only fill some part of the goals given. You can choose, as on the far left, to balance all three goals equally. In the next illustration, the focus has been moved to quick, with a resulting movement away from cheap and high quality.

一致性、可用性和分区容错性：选择两项

Consistency, Availability, and Partition Tolerance: Choose Two

在 2000 年计算机协会分布式计算原理研讨会上，Eric Brewer 发表了一篇题为“迈向鲁棒分布式系统”的论文。¹¹在本文中，Brewer 指出分布式系统不能很好地工作，因为应用程序要求信息在分布式数据库的所有副本之间保持一致。Brewer 认为，要实现真正的分布式计算，应用程序设计者必须放弃一致性。2002年，布鲁尔定理被证明是正确的，现在被称为CAP定理。CAP 定理简要指出，您不可能拥有一个一致、可用且具有分区容错性的数据库 — 您只能选择三个属性中的两个。为了更好地理解 CAP 定理在其本机环境中的情况，让我们更详细地了解一下所涉及的三个术语。

At the Association for Computing Machinery’s Symposium on the Principles of Distributed Computing in 2000, Eric Brewer presented a paper titled “Towards Robust Distributed Systems.”¹¹ In this paper, Brewer noted that distributed systems don’t work well because applications demand that the information be consistent among all copies of a distributed database. Brewer argued that to get to true distributed computing, application designers must give up consistency. In 2002, Brewer’s theorem was proven correct, and it is now known as the CAP theorem. The CAP theorem briefly states that you cannot have a database that is consistent, available, and exhibits partition tolerance—you can only choose two of the three properties. To better understand the CAP theorem in its native environment, let’s look at the three terms involved in a little more detail.

11 . Eric Brewer，“迈向稳健的分布式系统”，2000 年 7 月 19 日，np，2014 年 7 月 11 日访问，http://wisecracked/~brewer/cs262b-2004/PODC-keynote.pdf。

11. Eric Brewer, “Towards Robust Distributed Systems,” July 19, 2000, n.p., accessed July 11, 2014, http://wisecracked/~brewer/cs262b-2004/PODC-keynote.pdf.

•当数据库的任何用户无论何时何地读取相同的信息时，数据库都是一致的。例如，如果您将某件商品放入在线零售商的购物车中，则同一网站上的所有其他用户都应该看到购物车中的商品减少了库存。如果数据库不一致，那么两个人可以订购相同的商品，但只有一个人显示在库存中。一致性通常称为原子性；原子操作始终使数据库处于一种状态，即所有用户在每次操作后都具有相同的数据视图。

• A database is consistent when any user of a database will read the same information no matter when and where the read takes place. For instance, if you put an item in your shopping cart at an online retailer, every other user on that same site should see the inventory reduced by the item in your cart. If the database is not consistent, then two people can order the same item while only one shows as being in inventory. Consistency is often called atomicity; atomic operations always leave a database in a state where all users have the same view of the data after every operation.

•当没有用户被拒绝访问数据库进行读（或经常写）操作时，数据库是可访问的。虽然可访问性的定义通常是可变的，但一般来说，这意味着数据库不可访问的时间永远不会超过依赖数据库的外部进程或用户可以容忍的时间。

• A database is accessible when no user is refused access to the database for read (or often write) operations. While the definition of accessibility is often variable, in general, it means that the database is never inaccessible for longer than an outside process or user that relies on the database can tolerate.

• 当数据库可以分布在由网络分隔的多个设备和进程上而不影响数据库本身的整体操作时，它就可以容忍分区。根据定义，分布式数据库在包含数据库某些部分的每台机器上进行分区，并且通常需要分区来支持现实世界中数据库的性能和弹性要求。

• A database tolerates partitions when it can be spread across multiple devices and processes separated by a network without impacting the overall operation of the database itself. Distributed databases are, by definition, partitioned across each of the machines that contain some part of the database—and partitioning is often required to support the performance and resilience requirements of databases in the real world.

如图1.8所示，只需将标签替换为concentrate、accessible、partitionable，就可以用来说明CAP定理。

The diagram shown in Figure 1.8, by simply replacing the labels with consistent, accessible, and partitionable, can be used to illustrate the CAP theorem.

笔记

Note

CAP 定理在网络工程中的一个直接应用在于观察到路由协议只是分布式实时数据库系统。

One direct application of the CAP theorem to network engineering lies in the observation that routing protocols are simply distributed real-time database systems.

进入复杂性中心的旅程

Journey into the Center of Complexity

到目前为止，我们对复杂性的了解可以总结为以下几句话：

What we’ve learned about complexity to this point can be summarized in a few statements:

• 复杂性有一系列定义，围绕可理解性（包括意想不到的后果）、表面交互以及问题与用于解决问题的系统的关系等概念。

• Complexity has a range of definitions centering around the concept of comprehensibility (including unintended consequences), surface interactions, and the relationship of the problem to the system being used to solve the problem.

• 复杂性是对未来和现实世界双重不确定性的反应。

• Complexity is a reaction to the twin uncertainties of the future and the real world.

• 构建在有限的物理或虚拟空间内支持大量功能和功能的系统需要复杂性。

• Complexity is required to build systems supporting a lot of capabilities and functions within a restricted physical or virtual space.

• 复杂性是一组不可能对立的权衡，而不是单个“事物”。

• Complexity is a set of impossibly opposed tradeoffs, rather than a single “thing.”

为了进一步阐述最后一点，质量/速度/成本难题和 CAP 定理也可以成对而不是三个来看待。在任何一对相关项目中，两个项目之间都存在一条曲线，可以使用公式C ≤ 1/ R进行描述，如图 1.9所示，作为成本和脆弱性之间的权衡。这条曲线通常被称为图灵曲线。

To carry this last point further, the quality/speed/cost conundrum and the CAP theorem can be looked at in pairs rather than threes, as well. In any pair of related items, there is a curve between the two items that can be described using the formula C ≤ 1/R, illustrated in Figure 1.9, as a tradeoff between cost and fragility. This curve is often referred to as the Turing curve.

图 1.9 C ≤ 1/R 显示为脆弱性增加和成本增加之间的权衡

Figure 1.9 C ≤ 1/R Shown as a Tradeoff between Increasing Fragility and Increasing Cost

沿着这条曲线有一系列“最佳点”；工程师的工作是有意识或无意识、明智或不明智地选择曲线上的哪个点对每项权衡都有意义。

There is a range of “sweet spots,” along this curve; the job of the engineer is to choose which spot along the curve makes sense for every tradeoff—consciously or unconsciously, wisely or unwisely.

接下来的两章将更详细地研究复杂性的其他几个方面，特别是与计算机网络相关的方面。他们首先在下一章中研究复杂性的组成部分——组成网络的移动部件以及它们如何交互。第 3 章“测量网络复杂性”将研究测量网络复杂性的各种尝试，以检查该领域可用的工具。在这两章之后，本书将转向更实际的问题，目的是让网络工程师意识到他们在设计和部署协议以及设计和部署网络时所处理的复杂性权衡。当您走向复杂性的中心时，这个练习应该会带来更多的工程选择智慧。

The next two chapters investigate several more aspects of complexity in greater detail, particularly in relation to computer networks. They begin by examining the components of complexity in the next chapter—the moving parts that make up a network, and how they interact. Chapter 3, “Measuring Network Complexity,” will investigate various attempts at measuring complexity in a network to examine the tools available in this space. Following those two chapters, the book will turn to more practical matters, with the aim of making network engineers conscious of the complexity tradeoffs they are handling in designing and deploying protocols, and designing and deploying networks. This exercise should lead to more wisdom in engineering choices as you journey to the center of complexity.

第 2 章复杂性的组成部分

Chapter 2. Components of Complexity

在损坏的网络上工作始终是一项令人兴奋的工作，特别是如果您为一家大型供应商工作，每天都会有大量损坏的网络送到您家门口。为了在不断的攻击中生存下来，您最终开发了一组简单而快速的补丁或更改，您可以始终依赖这些补丁或更改来稳定网络，以便您可以开始实际解决问题的过程。例如，网络工程师在处理距离矢量控制平面中的故障时可能采取的一组步骤可以是：

Working on broken networks is always an exciting business to be in—particularly if you work for a large vendor where large broken networks are brought to your door every day. To survive the constant onslaught, you eventually develop a set of simple and quick patches or changes you can always rely on to settle a network down so you can start the process of actually troubleshooting the problem. For example, one set of steps network engineers dealing with failures in a distance-vector control plane might take to stabilize the network could be:

• 查看路由协议拓扑数据库。

• Look at the routing protocol topology database.

• 确定到达表中任何给定目的地的平均路径数。

• Determine how many paths, on average, there are to any given destination in the table.

• 将接口配置为被动接口（这样它们就不会交换可达性信息），直到并行路径的平均数量小于 4。

• Configure interfaces as passive (so they won’t exchange reachability information) until the average number of parallel paths is less than 4.

让我们再举一个例子：假设您正在检查一个不会收敛的网络，并且您注意到协议表中存在大量外部路由 - 假设表中 75% 或更多的路由是外部路由，并且它们似乎都年龄很短。为了稳定网络，您可能会采取的第一步是什么？找到重新分配点并用静态路由的重新分配替换协议之间的任何动态重新分配。

Let’s take another example: assume that you’re examining a network that won’t converge, and you notice a lot of external routes in the protocol tables—say 75% or more of the routes in the table are externals, and they all seem to have a very short age. What’s the first step you’re likely to take in stabilizing the network? Find the redistribution points and replace any dynamic redistribution between protocols with redistribution of static routes.

但是为什么这些技术应该作为启动和运行故障网络的初始步骤，以便可以采取进一步的故障排除和设计修复步骤呢？因为它们攻击所有三个主要组成部分大规模网络中的复杂性：状态数量、状态变化率以及交互面的范围。

But why should either of these techniques work as an initial step in getting a failed network up and running so further troubleshooting and design remediation steps can be taken? Because they attack all three of the major components of complexity in a large-scale network: the amount of state, the rate of state change, and the scope of interaction surfaces.

为了大规模构建和管理弹性网络，工程师需要管理复杂性。虽然这说起来很容易，但做起来却很难。与任何工程问题一样，第一步是决定如何设置问题。如何将问题分解为几个较小的部分，以便每个部分都可以单独管理？这些部分如何相互作用？通过考虑复杂性的这三个组成部分（状态、速度和表面）来检查复杂性问题，将有助于网络设计人员和架构师以有效且平衡的方式解决复杂性问题。虽然您可以从很多地方开始研究这个领域的复杂性，但控制平面融合是一个很好的起点，因为它涉及许多问题以及许多其他网络系统。

To build and manage resilient networks at scale engineers are going to manage complexity. While this is easy enough to say, it’s hard to do. As with any engineering problem, the first step is to decide how to set the problem up. How can the problem be broken into a few smaller pieces, so each one can be managed separately? How do these pieces interact? Examining the problem of complexity with these three components of complexity in mind—state, speed, and surfaces—will help network designers and architects address complexity in an effective and balanced way. While there are a number of places you could begin looking at the complexity in this space, control plane convergence is a good place to begin, because it touches many of the issues, and many of the other network systems.

网络融合

Network Convergence

网络融合是一个典型的例子，从中可以得出网络复杂性的各个组成部分。这是网络工程的一个领域，几乎每个在网络领域工作的人都以某种方式遇到过，并且复杂性的各个组成部分很容易作为复杂性领域内的单独概念来梳理和理解。

Network convergence is a prototypical example from which to draw the various components of network complexity. It’s an area of network engineering almost everyone who works in the networking field has encountered in some way, and the various components of complexity are fairly easy to tease out and understand as individual concepts within the realm of complexity.

路径向量：BGP 示例

Path Vector: A BGP Example

在边界网关协议（BGP）收敛的情况下，收敛时间的主要组成部分包括：

In the case of Border Gateway Protocol (BGP) convergence, the major components in convergence time include:

• 最小路由通告间隔(MRAI)：该计时器旨在防止状态变化淹没系统，特别是防止形成正反馈循环（请参阅第8 章“复杂系统如何失败”）。

• The Minimum Route Advertisement Interval (MRAI): This timer is designed to prevent state changes from overwhelming the system, particularly in preventing positive feedback loops from forming (see Chapter 8, “How Complex Systems Fail”).

• 处理和完成最佳路径计算所需的时间，特别是在路由服务器、路由反射器和其他处理比正常 BGP 路径集更大的设备中。

• The amount of time it takes to process and complete the best path calculations, particularly in route servers, route reflectors, and other devices that handle a larger than normal set of BGP paths.

• 任何特定发言者上的 BGP 进程与设备中的其他进程（例如路由信息库 (RIB) 和其他协议进程）交互所花费的时间。

• The amount of time the BGP process on any particular speaker spends interacting with other processes in the device, such as the Routing Information Base (RIB) and other protocol processes.

BGP 融合如图 2.1所示。

Figure 2.1 illustrates BGP convergence.

图2.1 BGP收敛

Figure 2.1 BGP Convergence

在该网络中，当Router F到2001:db8:0:2::/64的链路出现故障时：

In this network, when the link from Router F to 2001:db8:0:2::/64 fails:

• 路由器F 向路由器D、E 和B 发送撤回消息。路由器B 因为这是其到达2001:db8:0:2::/64 的最佳路径，所以将检查其到该目的地的可用路径。通过最佳路径过程，RouterB会选择经过RouterD的路径作为新的最佳路径，并向RouterA发送显式撤消的通告。

• Router F sends a withdraw to Routers D, E, and B. Router B, because this is its best path to 2001:db8:0:2::/64, will examine its available paths to this destination. Through the best path process, Router B will choose the path through Router D as the new best path, and send an advertisement with an explicit withdraw toward Router A.

• 接下来，路由器D 和E 完成对2001:db8:0:2::/64 的路径丢失的处理。RouterD向RouterB发送撤回消息；路由器E向路由器C发送撤回消息。

• Routers D and E finish processing the loss of the path to 2001:db8:0:2::/64 next. Router D sends a withdraw to Router B; Router E sends a withdraw to Router C.

• 路由器 B 在收到来自路由器 D 的撤销后，检查其表并确定现在最佳路径是沿路径 [C,E,F] — 请注意，此时路由器 C 仍在处理从路由器 E 收到的撤销因此RouterB仍然认为经过RouterC的路径是可用的。路由器 B 确定它应该向路由器 A 发送带有隐式撤销的新更新，但它现在必须等待 MRAI 超时才能发送。

• Router B, on receiving the withdraw from Router D, examines its table and determines that the best path is now along the path [C,E,F]—note that Router C is still processing the withdraw it received from Router E at this point, so Router B still believes that the path through Router C is available. Router B determines that it should send a new update with an implicit withdraw to Router A, but it must now wait for the MRAI to time out before it can.

• 路由器 C 现在完成对从路由器 E 收到的撤销的处理，并向路由器 B 发送撤销。

• Router C now finishes processing the withdraw it received from Router E, and sends a withdraw toward Router B.

• RouterB 检查其本地表，发现没有通往 2001:db8:0:2::/64 的路径。现在它向Router A 发送撤回消息，完成收敛过程。

• Router B examines its local table, and finds that it has no path toward 2001:db8:0:2::/64. It now transmits a withdraw to Router A, finishing the convergence process.

此示例展示了 BGP 在收敛时如何有效地从最短路径到最长路径工作。当 BGP 学习到新的目的地时，也会发生相同的情况。¹

This example shows how BGP effectively works from the shortest path to the longest when converging. The same situation occurs when BGP learns a new destination.¹

1 . Shivani Deshpande 和 Biplab Sikdar，“路由处理和 MRAI 计时器对 BGP 收敛时间的影响”，2012 年 4 月 27 日，np，2015 年 7 月 8 日访问，http: //www.ecse.rpi.edu/homepages/sikdar /papers/gbcom04s.pdf。

1. Shivani Deshpande and Biplab Sikdar, “On the Impact of Route Processing and MRAI Timers on BGP Convergence Times,” April 27, 2012, n.p., accessed July 8, 2015, http://www.ecse.rpi.edu/homepages/sikdar/papers/gbcom04s.pdf.

对于增加或减少自治系统 (AS) 路径的每个“周期”，MRAI 都会增加一个 MRAI 计时器收敛所需的时间。运行最佳路径的处理时间也会对 BGP 收敛所需的时间产生重大影响，特别是在 BGP 发言者必须处理大量路由（例如路由服务器或路由反射器）的情况下。运行最佳路径的时间还受到 BGP 进程与路由器上运行的其他进程（例如 RIB 进程）的交互的影响。构建可靠的 BGP 实现并不是一件容易的事——世界上只有少数可靠的、广泛使用的 BGP 实现。

The MRAI increases the time required to converge by one MRAI timer for each “cycle” of increasing or decreasing the autonomous system (AS) Path. The processing time of running best path can also have a major impact on the time required for BGP to converge, especially in cases where the BGP speaker must process a large number of routes (such as a route server or route reflector). The time to run best path is also impacted by the interaction of the BGP process with other processes running on the router, such as the RIB process. Building a solid BGP implementation is not an easy task—there are only a handful of solid, widely used, BGP implementations in the world.

距离向量：EIGRP 示例

Distance Vector: An EIGRP Example

虽然增强型内部网关协议 (EIGRP) 的使用不再像以前那样广泛，但仍然值得研究一下 EIGRP 融合过程，以深入了解分布式控制平面的融合方式。图 2.2说明了用于讨论 EIGRP 收敛的网络。

While Enhanced Interior Gateway Protocol (EIGRP) isn’t as widely used as it once was, it’s still worth looking at the EIGRP convergence process to gain a solid understanding of the way distributed control planes converge. Figure 2.2 illustrates a network used for discussing EIGRP convergence.

图 2.2 EIGRP 收敛处理

Figure 2.2 EIGRP Convergence Processing

这个过程相当简单：

This process is fairly simple:

1.路由器 D 发现它已丢失与 2001:db8:0:2::/64 的链路，并检查其本地表以查找备用路由。如果没有找到，它会向路由器 C 发送查询，以发现路由器 C 是否有备用路由。路由器 D在等待对此查询的响应时，将到 2001:db8:0:2::/64 的路由置于活动状态。

1. Router D discovers that it has lost its link to 2001:db8:0:2::/64 and examines its local table for an alternate route. Finding none, it sends a query to Router C, to discover if Router C has an alternate route. Router D places the route to 2001:db8:0:2::/64 in the active state while it awaits the response to this query.

2.路由器 C 接收此查询并检查其本地表以确定是否有替代路由（除了通过路由器 D 之外）。如果没有找到，它就会向路由器 B 发送查询，并将到 2001:db8:0:2::/64 的路由置于活动状态，同时等待对此查询的响应。

2. Router C receives this query and examines its local table to determine if it has an alternate route (other than through Router D). Finding none, it will then send a query to Router B, and place the route to 2001:db8:0:2::/64 into the active state while it waits for the response to this query.

3.路由器 B 收到此查询并检查其本地表以确定是否有备用路由（除了通过路由器 C 之外）。如果没有找到，它将向路由器 A 发送查询，并将到 2001:db8:0:2::/64 的路由置于活动状态，同时等待对此查询的响应。

3. Router B receives this query and examines its local table to determine if it has an alternate route (other than through Router C). Finding none, it will then send a query to Router A, and place the route to 2001:db8:0:2::/64 into the active state while it waits for the response to this query.

4.路由器 A 收到此查询，并且找不到备用路由，也没有找到任何可以询问该目的地的其他邻居，因此从其本地路由表中删除该路由并向路由器 B 发送回复。

4. Router A receives this query and, finding no alternate route, nor any other neighbors that it can ask about this destination, removes the route from its local routing table and sends a reply to Router B.

5.路由器 B 收到此回复，从其本地路由表中删除该目的地，并向路由器 C 发送回复。

5. Router B receives this reply, removes the destination from its local routing table, and sends a reply to Router C.

6. Router C 收到 Router B 的回复，从本地路由表中删除该目的地，并向 Router D 发送回复。

6. Router C receives the reply from Router B, removes the destination from its local routing table, and sends a reply to Router D.

7. RouterD 收到 RouterC 的回复后，将 2001:db8:0:2::/64 从本地路由表中删除。

7. Router D receives the reply from Router C and removes 2001:db8:0:2::/64 from its local routing table.

这可能看起来需要大量工作（最坏的情况是有意从处理速度的角度来说明该过程），但每个路由器都可以相当快地处理查询或回复，因为每个步骤几乎没有什么可做的。事实上，在“野外”运行的典型 EIGRP 网络中，每个查询跃点收敛所需的平均时间约为 200 毫秒。

This might seem like a lot of work (and the worst possible case has intentionally been to illustrate the process from a speed of processing perspective), but each router can process the query or reply fairly rapidly, because there is little to do for each step. In fact, in typical EIGRP networks running “in the wild,” the average amount of time required to converge per query hop is around 200 milliseconds.

尽管如此，很容易看出正在处理的状态量如何对收敛所需的时间产生重大影响，从而对网络的稳定性产生重大影响，因为每个路由器必须串行处理有关网络拓扑修改的信息。方式。例如，路由器 A 无法完全处理它所拥有的与 2001:db8:0:2::/64 连接丢失的信息，直到查询范围内的所有其他路由器都已经处理了该信息。

Nonetheless, it is easy to see how the amount of state being processed could have a major impact on the time it takes to converge, and hence the stability of the network, as each router must process information about modifications in the network’s topology in a serial way. Router A cannot, for instance, fully process the information it has about the loss of connection to 2001:db8:0:2::/64 until every other router within the scope of the query has already processed this information.

虽然每个路由器可能只需要 200 毫秒来处理拓扑更改，但具有数百或数千个更改的单个事件将导致查询路径中的每个路由器单独处理每个可到达目的地的更改（非常类似于 BGP）。因此，可达性的大规模变化可能会给受查询影响的每个设备的处理器和存储带来很大的压力。

While each router might only need 200 milliseconds to process the topology change, a single event with hundreds or thousands of changes will cause each router in the query path to process the change for each reachable destination separately (much like BGP). Hence a large-scale change in reachability can put a good deal of stress on the processor and storage for every device impacted by the query.

链路状态：OSPF 和 IS-IS 融合

Link State: OSPF and IS-IS Convergence

链路状态协议（例如 OSPF 和 IS-IS）具有不同的收敛属性集；图 2.3说明了这个过程。

Link state protocols, such as OSPF and IS-IS, have a different set of convergence attributes; Figure 2.3 illustrates this process.

图2.3 链路状态收敛处理

Figure 2.3 Link State Convergence Processing

图示步骤如下：

The steps illustrated are:

1. RouterD发现2001:db8:0:2::/64不再可达。为了响应网络拓扑的这种变化，它会构建一个链路状态通告（LSA，对于OSPF），或者重建其链路状态协议数据单元（PDU；LSP是链路状态数据包，它类似于OSPF中的LSA） OSPF），并向路由器 C 通告此新信息。

1. Router D discovers that 2001:db8:0:2::/64 is no longer reachable. In response to this change in the network topology, it will build a Link State Advertisement (LSA, for OSPF), or rebuild its Link State Protocol Data Unit (PDU; an LSP is a Link State Packet, which is similar to an LSA in OSPF), and advertise this new information toward Router C.

2.路由器 C 在收到此新信息后，将简单地将副本转发给其邻居路由器 B，而不处理该信息。路由器 C 最终将处理此信息，但链路状态协议通常首先泛洪，然后再处理，以提高参与控制平面的所有设备的数据库同步的速度。

2. Router C, on receiving this new information, will simply forward a copy along to its neighbor, Router B, without processing the information. Router C will eventually process this information, but link state protocols typically flood first, and process later, to increase the speed at which the databases of all the devices participating in the control plane will be synchronized.

3.路由器 B 在收到此新信息后，将简单地将副本转发给其邻居路由器 A，而不处理该信息。

3. Router B, on receiving this new information, will simply forward a copy along to its neighbor, Router A, without processing the information.

4.在稍后的某个时间点（由链路状态协议内的计时器设置），路由器 D 将运行本地最短路径优先 (SPF) 计算，以确定本地路由表中需要更改的内容。结果将是路由器 D 从其本地路由表中删除 2001:db8:0:2::/64。

4. At some point later in time (set by a timer within the link state protocol), Router D will run a local Shortest Path First (SPF) computation to determine what needs to be changed in the local routing table. The result will be Router D removing 2001:db8:0:2::/64 from its local routing table.

5.在路由器 D 计算出新的最短路径树 (SPT) 后不久，路由器 C 也会执行同样的操作，通过删除 2001:db8:0:2::/64 来调整其本地路由表。

5. Shortly after Router D computes a new Shortest Path Tree (SPT), Router C will do likewise, adjusting its local routing table by removing 2001:db8:0:2::/64.

6.在路由器 C 之后不久，路由器 B 将执行相同的计算，并得到相同的结果。

6. Shortly after Router C, Router B will perform the same computation, with the same results.

7.最后，路由器A将执行相同的计算，并得到相同的结果。

7. Finally, Router A will perform the same computation, with the same results.

笔记

Note

这是汇聚链路状态控制平面所需处理的稍微简化的视图；本书稍后将在复杂性的背景下考虑更多细节。读者还可以查看IS-IS for IP Networks ²等书籍，更深入地了解链路状态处理。

This is a somewhat simplified view of the processing required to converge a link state control plane; more detail will be considered in the context of complexity later in this book. Readers can also look at books such as IS-IS for IP Networks² to understand link state processing more deeply.

2 . Russ White 和 Alvaro Retana，IS-IS：IP 网络中的部署，第一版。（波士顿：艾迪生韦斯利，2003 年）。

2. Russ White and Alvaro Retana, IS-IS: Deployment in IP Networks, 1st edition. (Boston: Addison-Wesley, 2003).

在这种情况下，数据包中携带的状态量会影响网络拓扑更改的处理时间：

In this case, the amount of state being carried in the packet impacts the processing time for a network topology change in:

• 网络中从一个路由器到另一个路由器传输链路状态信息所花费的时间。这将包括将数据包序列化到线路上、对数据包从线路上计时、对数据包进行排队等。任何需要多个数据包以这种方式在网络上泛洪的网络拓扑更新必然需要更长的时间。描述网络拓扑变化所需的状态越多，携带该信息所需的数据包数量就越多。

• The amount of time it takes to transmit the link state information from router to router in the network. This would include serializing the packet onto the wire, clocking the packet off the wire, queuing the packet, etc. Any network topology update that requires more than one packet to flood in this way across the network will necessarily take longer. The more state required to describe the changes in the network topology, the larger the number of packets required to carry that information.

• 处理网络拓扑更改所需的时间、处理能力和内存量将取决于更改的数量或现有状态的数量。当然，有多种方法可以优化此处理（例如部分 SPF），几乎达到这样的程度：附加状态通常只能对收敛时间产生微不足道的影响。

• The amount of time, processing power, and memory it takes to process the changes to the network topology will depend on the number of changes, or the amount of existing state. There are, of course, ways to optimize this processing (such as partial SPFs), almost to the point that the additional state can often only have a trivial effect on the time it takes to converge.

对于链路状态协议来说，当网络拓扑发生变化时，协议中携带的大量状态与路由协议收敛所需的时间之间仍然存在联系。

For link state protocols, there is still a connection between the sheer amount of state carried in the protocol and the time required for the routing protocol to converge when a change in the network topology occurs.

状态

State

在大型系统中，状态的绝对数量可能是巨大的——不仅对于在系统上工作的人员来说，而且对于必须管理和处理信息的协议和计算机系统来说也是如此。让我们看看状态量很重要的一些原因。

In large-scale systems, the sheer amount of state can be overwhelming—not only for the people working on the system, but also for the protocols and computer systems that must manage and process the information. Let’s look at some of the reasons why the amount of state matters.

信息量

Amount of Information

第一个因素是需要通过网络传输才能收敛的信息量。

The first of these factors is the amount of information that needs to be transferred across the network to converge.

暂时考虑一下封装单个 BGP 更新所包含的信息量。根据数据包格式和历史信息，假设单个 BGP 更新消耗大约 1500 字节（八位组）的内存。截至撰写本文时，当前完整表大小超过 50 万个目的地，这将需要至少 795MB 的空间必须在网络中的路由器之间传递。这不包括 TCP 开销、用于格式化数据的 TLV 以及其他传输要求。

Consider, for a moment, the amount of information contained in encapsulating a single BGP update. Based on packet format and historical information, assume that a single BGP update consumes about 1500 bytes (octets) of memory. At the time of this writing, the current full table size is over a half a million destinations, which will require at least 795MB that must be passed around between the routers in the network. This doesn’t include TCP overhead, TLVs for formatting the data, and other transport requirements.

在 5GB 演示文稿的世界中，795MB 的数据可能看起来不多，但请记住这是一个在大量路由器上运行的分布式数据库。多少？截至撰写本文时，大约有 48,000 个 AS 连接到互联网。³任何给定AS中的BGP发言者数量可以在十到数千之间；由于没有真正的方法可以知道单个 AS 中发言者的平均数量是多少，我们使用保守的估计，并将其称为每个 AS 10 个路由器。根据这些数字，这个 795MB 的表正在大约 480,000 台设备之间同步。现在听起来不那么小了，是吗？

795MB of data might not seem like a lot in a world of 5GB presentations, but remember this is a distributed database running on a very large number of routers. How many? There are around 48,000 AS connected to the Internet at the time of this writing.³ The number of BGP speakers in any given AS can be between ten and thousands; given there’s no real way to know what the average number of speakers in a single AS is, let’s use a conservative estimate, and call it 10 routers per AS. With these numbers, this 795MB table is being synchronized between some 480,000 devices. Doesn’t sound so small now, does it?

3 . “Cymru 团队互联网监控 — BGP 唯一 ASN 计数”，np，2014 年 8 月 24 日访问，http://www.cymru.com/BGP/unique_asns.html。

3. “Team Cymru Internet Monitor—BGP Unique ASN Count,” n.p., accessed August 24, 2014, http://www.cymru.com/BGP/unique_asns.html.

这可能令人印象深刻，但普通网络并不是互联网。即便如此，由 1000 个路由器组成的网络仍然可以近乎实时地在这 1000 个路由器之间保持数兆字节的表同步——以秒或毫秒为单位，而不是几分钟、几小时或几天。

This might all be impressive, but the average network isn’t the Internet. Even so, a network of 1000 routers is keeping a multi-megabyte table synchronized across those 1000 routers in near real time—in seconds or milliseconds, rather than minutes, hours, or days.

对于距离矢量协议，更新中携带的信息量也是一个因素，但方式不同。例如，运行距离矢量协议的网络中的每个并行链路代表由这些并行链路连接的两个设备之间携带的可达性信息的全新副本。这些额外的信息副本有时可能会变得不同步，或者遇到导致可达性视图不一致的其他问题。这些额外的并行链路还代表潜在的反馈循环，可能导致网络永远无法收敛。因此，在上面的示例中，网络工程师可能会关闭并行链路以稳定距离矢量控制平面。这可以从网络中删除足够的附加状态，以允许控制平面完全收敛，

For distance-vector protocols, the amount of information carried in the updates is also a factor, but in a different way. For instance, each parallel link in a network running a distance-vector protocol represents a completely new copy of the reachability information being carried between the two devices connected by these parallel links. These additional copies of the information can sometimes become desynchronized, or encounter other problems that cause an inconsistent view of reachability. These additional parallel links also represent potential feedback loops that can cause the network to never converge. Hence, in the example above, the network engineer might shut down parallel links to stabilize a distance-vector control plane. This can remove enough additional state from the network to allow the control plane to fully converge, bringing the network back into operation while a deeper analysis of the problems that brought the failure about can be investigated.

链路状态更新中携带的信息量以大致相同的方式影响链路状态协议的操作。信息在网络中传播不仅需要更长的时间，而且还会建立一个更大的数据库，需要在该数据库上运行 SPF 算法来构建一致的网络拓扑视图。如果附加信息位于最短路径经过的节点中，而不是仅沿着网络边缘离开，则附加信息会直接影响 SPF 运行的速度。

The amount of information carried in link state updates impacts the operation of a link state protocol in much the same way. Not only does it take longer for the information to be flooded through the network, it also builds a bigger database across which the SPF algorithm needs to be run to construct a consistent view of the network topology. If the additional information is in nodes through which shortest paths pass, rather than just leaves along the edge of the network, the additional information can impact the speed at which SPF runs directly.

现实世界中国家失败的一个例子

An Example of State Failure in the Real World

大多数网络故障并不是纯粹的“状态驱动”事件；几乎总是涉及状态、速度和表面的某种组合。然而，有一些几乎纯粹是状态驱动的，例如大型中心辐射网络中的链路反弹导致的网络崩溃。图 2.4提供了一个可供讨论的基本网络。

Most network failures are not pure “state-driven” events; there is almost always some combination of state, speed, and surface involved. There are a few, however, that are almost purely state driven, such as network meltdowns caused by a link bounce in a large-scale hub-and-spoke network. Figure 2.4 provides a basic network for discussion.

图 2.4 中心辐射型网络故障示例

Figure 2.4 Hub and Spoke Network Failure Example

该网络最初只有几个辐条或远程站点，并随着时间的推移而增长。随着新辐条的添加，状态数量不断增加；只要状态量缓慢攀升，控制平面就有足够的机会来适应可达性的微小变化。然而，路由器 A 的多点链路故障会导致整个分布式数据库立即被修改，这往往会超出控制平面的处理能力。考虑以下事件顺序：

This network starts out with just a few spokes or remote sites, and grows over time. As new spokes are added, the amount of state climbs; so long as the amount of state climbs slowly, the control plane has ample opportunity to adjust to the small changes in reachability. A failure of the multipoint link at Router A, however, causes the entire distributed database to be revised at once—often overwhelming the ability of the control plane to cope. Consider the following sequence of events:

• RouterA 的链路发生故障，导致所有邻居邻接关系同时发生故障。

• The link at Router A fails, causing all the neighbor adjacencies to fail at the same time.

• 链路恢复，导致连接到中心辐射网络的所有路由器尝试与中心路由器路由器 A 建立邻接关系。

• The link is recovered, causing all of the routers connected to the hub and spoke network to attempt building an adjacency with Router A, the hub router.

• 一些分支路由器成功开始与路由器 A 形成邻接关系。

• Some number of spoke routers successfully begin to form an adjacency with Router A.

• 已开始与路由器A 成功形成邻接关系的分支路由器将其完整的路由表发送至集线器，以完成这些邻接关系的形成。此信息淹没了路由器 A 的输入队列，导致 hello 和其他邻接形成数据包被丢弃。

• The spoke routers that have begun to successfully form an adjacency with Router A send their complete routing tables toward the hub to complete the formation of these adjacencies. This information overwhelms the input queue at Router A, causing hello and other adjacency formation packets to be dropped.

• 这些丢弃的数据包导致邻接形成过程中止，从而重新开始循环。

• These dropped packets cause the adjacency formation process to abort, restarting the cycle.

解决这种不断尝试形成大量邻接而导致所有邻接失败、导致在短时间内又一轮尝试形成大量邻接的可能方法是减慢该过程。首先只允许少量的分支路由器形成邻接。一旦一组邻接成功形成，就允许另一组分支路由器形成邻接。将允许形成邻接的分支路由器集分解为小组可以控制整个网络的信息流，使其低于中心路由器可以处理的水平。请注意，将邻接结构分解成小组模拟了网络最初构建的过程——随着时间的推移，分成更小的块。

One possible way to resolve this constant attempt at forming a large number of adjacencies that cause all the adjacencies to fail, causing another round of attempting to form a large number of adjacencies in a short period of time, is to slow the process down. Start by allowing only a small number of spoke routers to form an adjacency. Once one set of adjacencies is formed successfully, allow another set of spoke routers to form an adjacency. Breaking the set of spoke routers allowed to form an adjacency into small groups controls the information flow across the network, keeping it below the level the hub router can process. Note that breaking the adjacency formation down into small groups emulates the process by which the network was built in the first place—in smaller chunks, over time.

关于国家的最终想法

Final Thoughts on State

如果将路由协议视为分布式、近实时的数据库，那么数据库收敛所需的时间实际上就是数据库不一致的时间。EIGRP 的示例尤其令人心酸：EIGRP 活动计时器是您愿意允许网络保持不收敛的时间量，因此（在 EIGRP 的特定情况下）是您愿意允许数据包通过的时间长度。被丢弃而不是转发到最终目的地。BGP 与此类似，尽管流量通常不是最优路由而不是丢弃，并且抖动和延迟结果符合预期。对于链路状态协议，称为控制平面的分布式数据库保持不一致的时间量是流量可以循环或丢弃的时间量（取决于拓扑更改的类型和处理顺序）。状态可以分解为更小的块，以便更有效地处理，如中心和分支网络故障的示例所示。

If the routing protocol is viewed as a distributed, near real-time database, then the amount of time it takes for the database to converge is actually the amount of time the database is inconsistent. The example from EIGRP is particularly poignant: the EIGRP active timer is the amount of time you’re willing to allow your network to remain unconverged, and hence (in the specific case of EIGRP), how long you’re willing to allow packets to be dropped rather than forwarded to their final destination. BGP is similar to this, although traffic is more often routed suboptimally rather than dropped, with the expected results on jitter and delay. For a link state protocol, the amount of time the distributed database called the control plane remains inconsistent is the amount of time traffic can either be looped or dropped (depending on the type of topology change and the order of processing). State can be broken up into smaller chunks to be dealt with more efficiently, as shown in the example of the hub and spoke network failure.

速度

Speed

在大多数情况下，变化的速度实际上比系统中的状态数量更能预测网络故障。只要状态相对静态，就不会产生“在线”成本或在网络节点上进行处理；静态状态主要是转发（或数据）平面而不是控制平面的成本。让我们看一下状态变化速度对网络收敛产生重大影响的两个示例。

The speed of change is, in most cases, actually a stronger predictor of network failure than the sheer amount of state in the system. So long as the state is relatively static, it isn’t costing “on the wire” or processing on the network nodes; the static state is mostly a cost in the forwarding (or data) plane rather than in the control plane. Let’s look at two examples of speed of state change having a major impact on network convergence.

永不融合的网络

The Network That Never Converges

让我们从一个简单的问题开始：全球互联网需要多长时间才能融合？换句话说，如果您从上游提供商的网络中的某个随机边缘对等点删除一条路由，“互联网的其余部分”需要多长时间才能发现该目的地不再可达？为了使问题更简单，我们假设该路由在第 3 层提供商的边缘被删除，而不是在第 1 层提供商的边缘，因此该路由必须传播到四个 AS 才能“随处”被删除（实际上，跳数由于 AS 跳数的长尾分布，会比这个长，但此示例将坚持使用四跳，因为它是接近平均值的整数）。

Let’s begin with a simple question: how long does it take for the global Internet to converge? In other words, if you remove a route from some random edge peering point, from an upstream provider’s network, how long will it for “the rest of the Internet” to discover that this destination is no longer reachable? To make the problem simpler, let’s assume that this route is removed at the edge of a tier 3 provider, rather than a tier 1 provider, so the route must propagate across four AS to be removed “everywhere” (in reality, the hop count would be longer than this because of the long tail distribution of AS hop count, but this example will stick with four hops here because it’s a round number close to the average).

那么，收敛时间问题就变成了“一条 BGP 路由需要多长时间才能传播通过四个自治系统？” 基于大量研究和实验室工作，答案实际上相当简单：在没有路由衰减的情况下，BGP（大致）根据以下公式收敛：

The convergence time question becomes, then—“how long does a BGP route take to propagate through four autonomous systems?” The answer is actually fairly simple, based on a lot of research and lab work: given no route dampening, BGP (roughly) converges based on the formula:

收敛 =（最大 AS 路径长度 – 最小 AS 路径长度）*MRAI

Convergence = (Max AS Path Length – Min AS Path Length)*MRAI

其中 MRAI 是最小路由通告间隔，或者是在通告有关特定目的地的信息之后、通告有关同一目的地的最新信息之前的时间量。如果该路由最初具有 4 跳的 AS 路径，并且该路由现在无法到达，并且 MRAI 为 30 秒，那么根据经验，大约需要 2 分钟才能从全局表中删除该路由。

where MRAI is the minimum route advertisement interval, or the amount of time after advertising information about a particular destination before advertising more recent information about that same destination. If the route originally had an AS Path of 4 hops, and the route is now unreachable, and the MRAI is 30 seconds, then it will take, as a rule of thumb, around 2 minutes to remove the route from the global table.

如果路由被注入而不是被删除怎么办？此案例涉及整个全球互联网中每个 BGP 发言者（内部和外部、端到端）接收新路由信息、处理该信息，然后将其发送到其对等方所需的时间。MRAI 影响此过程，如图2.5所示。

What if the route is injected instead of being removed? This case deals with the time it takes for each BGP speaker—both internal and external, end to end in the entire global Internet—to receive the new routing information, process it, and then send it on to its peers. The MRAI impacts this process as illustrated in Figure 2.5.

图 2.5 用于传播新目的地的 BGP 收敛

Figure 2.5 BGP Convergence for Propagating a New Destination

在此网络中，假设 BGP 通告的顺序可能最差，当 2001:db8:0:2::/64 首次连接到路由器 F 时：

In this network, assuming the worst possible ordering of BGP advertisements, when 2001:db8:0:2::/64 is first connected to Router F:

1. RouterF将新的目的地发布给RouterE，RouterE再将这条新路由发布给RouterC。RouterC将这条新路由发布给RouterB，RouterB再发布给RouterA。此时，RouterB设置MRAI为这个目的地。

1. Router F advertises the new destination to Router E, which then advertises this new route to Router C. Router C advertises this route to Router B, which then advertises the route to Router A. At this point, Router B sets the MRAI for this destination.

2.路由器 F 还将这个新目的地通告给路由器 D，然后路由器 D 将同一目的地通告给路由器 B。该通告在路由器 C 通告之后不久到达，并赢得了路由器 B 的最佳路径计算。路由器 B，但是，在 MRAI 定时器到期之前，无法将这条新路由通告给 RouterA。一旦 MRAI 定时器到期，路由器 B 就会向路由器 A 通告这条较短的路径。

2. Router F also advertises this new destination to Router D, which then advertises the same destination to Router B. This advertisement arrives just a few moments after the advertisement from Router C, and wins the best path calculation at Router B. Router B, however, cannot advertise this new route to Router A until the MRAI timer expires. Once the MRAI timer expires, Router B advertises this shorter path to Router A.

3. RouterF 也直接向RouterB 发布2001:db8:0:2::/64。该通告在 MRAI 定时器到期后立即到达路由器 B，路由器 B 已将经过路由器 D 的路径通告给路由器 A，并重置了 MRAI 定时器。路由器 B 现在必须等待 MRAI 计时器（再次）到期，然后才能向路由器 A 通告新的（且更短的）路径。

3. Router F also advertises 2001:db8:0:2::/64 to Router B directly. This advertisement reaches Router B just moments after the MRAI timer expires, Router B has advertised the path through Router D to Router A, and has reset the MRAI timer. Router B must now wait until the MRAI timer expires (again) before it can advertise the new (and shorter) path to Router A.

这一系列事件已在实时网络（例如全球互联网）中观察到。在这种情况下，MRAI 导致新到达目的地的通告需要几分钟，而不是几秒钟。

This sequence of events has been observed in live networks (such as the global Internet). The MRAI, in this instance, causes the advertisement of newly reachable destinations to take minutes, rather than seconds.

现在，另一方面，全球互联网多久发生一次变化？图 2.6是在撰写本文时取自potaroo.net的图表，该网站测量全局路由表的状态。⁴

Now, on the other side, how often do changes happen on the global Internet? Figure 2.6 is a chart taken from potaroo.net, a site that measures the state of the global routing table, at the time of writing.⁴

4 . “BGP 不稳定报告”，np，2014 年 8 月 24 日访问，http://bgpupdates.potaroo.net/instability/bgpupd.html。

4. “The BGP Instability Report,” n.p., accessed August 24, 2014, http://bgpupdates.potaroo.net/instability/bgpupd.html.

图2.6 全球互联网路由表变化率

Figure 2.6 Rate of Routing Table Change in the Global Internet

以下是一些观察结果：

Several observations are in order:

• 纵轴表示全局（默认自由区域）互联网路由表每秒的变化率。平均每秒变化 15 到 30 次，峰值最高可达每秒 50 次变化。

• The vertical axis represents the rate of change in the global (default free zone) Internet routing table per second. The average seems to be between 15 and 30 changes per second, with peaks that reach as high as 50 changes per second.

• 水平轴代表时间，并在此数据呈现中按跨天发生的情况进行分割。检查多年来测量的可用信息表明，这种模式存在多年。

• The horizontal axis represents time, and is split up in this rendition of the data as occurring across days. Examining the information available across years of measurements indicates that this pattern exists across many years.

从相同的数据可以看出，全球互联网表中任何特定变化的收敛速度都是以秒或分钟为单位来衡量的（70 到 80 秒似乎是平均值）。

From the same data, it’s apparent that the rate of convergence for any particular change in the global Internet table is measured in seconds or minutes (70 to 80 seconds would seem to be the average).

很难将路由表中每秒发生 15 到 50 次更改的网络称为真正意义上的收敛。事实上，互联网从未真正融合——而且已经很多年没有真正融合了。那么，互联网控制平面如何提供足够可靠的可达性信息以使互联网本身正常运行呢？为什么互联网的控制平面不会像任何正常网络中所预期的那样“崩溃”？互联网控制平面的稳定性主要归因于“核心”的相对稳定性（将大部分状态变化移至网络边缘）以及任何特定 AS 中内部和外部路由信息之间的严格划分。后一点将在几页的“表面”部分中进行讨论。

It’s difficult to call a network with 15 to 50 changes per second in its routing table converged in any meaningful sense of the word. In fact, the Internet doesn’t ever really converge—and it hasn’t really converged in years. How, then can the Internet control plane provide reachability information reliable enough to make the Internet itself work? Why doesn’t the Internet’s control plane “crash,” as might be expected in any normal network? Primarily the stability of the Internet’s control plane is due to the relative stability of the “core,” which moves most state changes to the edges of the network, and the strong division between internal and external routing information in any specific AS. This latter point is considered in the section, “Surfaces,” in a few pages.

摆动的链接

The Flapping Link

振荡链路不再像以前那样常见，尤其是广域链路，但它们对收敛的影响可能是毁灭性的。图 2.7给出了一个网络供参考。

Flapping links are not as common as they used to be, particularly wide area links, but they can be devastating in their impact on convergence. Figure 2.7 illustrates a network for reference.

图 2.7 链路抖动和基于速度的网络故障

Figure 2.7 Link Flaps and Speed-Based Network Failure

对于路由器 A 和 B 之间的链路的每次抖动，路由器 B、C、D 和 E 都会收到数千个更新。然而，路由器 F 在同一时间段内始终收到三倍多的更新——这是众多缺陷之一具有分布式控制平面的大规模并行拓扑。在实际情况中，路由器 F 可能会发生故障，导致网络无法收敛。根据该网络中故障域的配置和范围，路由器 F 的故障（或者甚至无法跟上不断更新的拓扑信息流）可能会渗透到路由器 F 之外的网络中，从而导致一般控制平面故障。

For each flap of the link between Routers A and B, Routers B, C, D, and E receive thousands of updates. Router F, however, consistently receives three times as many updates in the same time period—one of the many downfalls of massively parallel topologies with distributed control planes. In real-world situations, Router F could fail in a way that prevents the network from ever converging. Depending on the configuration and scope of the failure domains in this network, Router F’s failure (or even its inability to keep up with this constant flow of updated topology information) could bleed into the network beyond Router F, causing a general control plane failure.

振荡链路的速度与可倍增拓扑更新速度的并行链路的结合对于路由协议来说可能是致命的。

The combination of the speed of a flapping link and parallel links that multiply the speed of topology updates can be fatal to a routing protocol.

关于速度的最终想法

Final Thoughts on Speed

杀死控制平面的并不是变化的速度，而是信息变化速度的不可预测性以及每个时间片中变化的信息量。变化率越随机，变化的信息量越随机，围绕变化进行计划就越困难。大多数网络工程师在设计网络时都会考虑事物互连的方式、服务应放置的位置以及如何使事情对操作人员来说“更简单”。通常不考虑控制平面的稳定性；这只是一个“给定”。

It’s not so much the speed of change that kills control planes, it’s the unpredictability of the speed at which information changes combined with the amount of information changing in each time slice. The more random the rate of change, and the more random the amount of information changing, the harder it is to plan around the changes. Most network engineers, when designing a network, consider the way in which things should be interconnected, where services should be placed, and how to make things “simpler” for the human operator. What isn’t often considered is the stability of the control plane; it’s just a “given.”

处理网络复杂性时，速度是需要考虑的关键点；更快通常意味着更复杂。

Speed is a crucial point to consider when dealing with network complexity; faster generally means more complex.

表面

Surface

还有一个想法与状态量和变化速度混合在一起，要么放大要么减弱两者，从而影响控制平面的稳定性：不同组件或系统相互作用的表面。理解复杂系统中的交互表面涉及三个基本概念：图 2.8说明了前两个。

There is one more idea that mixes with the amount of state and the speed of change, either amplifying or dampening both, and hence impacting control plane stability: the surfaces across which different components or systems are interacting. Three basic concepts are involved in understanding interaction surfaces in complex systems; Figure 2.8 illustrates the first two.

图 2.8 交互界面

Figure 2.8 Interaction Surfaces

交互界面有两个基本概念：

There are two basic concepts in interaction surfaces:

•互动深度。交互深度可以看作两个系统或组件交互的强度。例如，如果一个组件依赖于另一个组件以特定方式格式化数据，则这两个组件必须并行地更改特定数据的格式化方式。当格式化数据的组件发生变化时，需要知道数据如何格式化的组件也必须发生变化。在网络架构领域，这可以看作是两个不同控制平面之间的交互，或者是在流经网络时检查数据包的格式。单个数据包格式更改可能会导致数百或数千个设备需要更新才能正确读取新的数据包格式。图 2.8中的 A 点和 C 点说明所示的两个组件之间的深层交互。

• Interaction Depth. The depth of interaction can be seen as how strongly the two systems or components interact. For instance, if one component relies on another component formatting data in a specific way, the two components must change in terms of the way that specific piece of data is formatted, in parallel. As the component that formats the data is changed, the component that needs to know how the data is formatted must also change. In the world of network architecture, this can be seen as the interaction between two different control planes, or the formatting of packets being inspected as they flow through the network. A single packet change format can cause hundreds or thousands of devices to require updates to read the new packet format correctly. Points A and C in Figure 2.8 illustrate deep interaction between the two components shown.

•互动广度。两个系统或组件“接触”的地方的数量可以称为交互表面的宽度。两个系统或组件交互的地方越多，它们就越会形成一个更复杂的系统。在图2.8中，两个所示系统有三个接触点；其中一个（C 点）比其他两个更宽，表示沿单个任务（或一组任务）定位的多个接口。

• Interaction Breadth. The number of places where two systems or components “touch” can be called the breadth of the interaction surface. The more places two systems or components interact, the more they will form a single, more complex system. In Figure 2.8, there are three points at which the two illustrated systems touch; one of these (point C) is wider than the other two, representing a number of interfaces located along a single task (or set of tasks).

可以使用一个示例来说明这两个概念：在整个网络的一组路由器上配置两个路由协议。

A single example can be used to illustrate both concepts: two routing protocols configured on a set of routers throughout a network.

• 单个路由器上配置的每个协议都通过共享资源（例如内存和处理器）与同一路由器上的任何其他协议进行交互。安装在单个路由器上的多个协议也共享一个公共 RIB（或一组 RIB），因此一个协议删除路由可能会导致第二个协议做出反应 — 也许会通告一个路由替换路线，或删除只能通过（现已删除）目的地到达的目的地的可达性。第一组交互——共享资源的竞争——通常可以被认为是范围狭窄的浅层交互表面。第二组交互（通过共享 RIB 共享和交互可达性信息）可以被视为稍宽但仍然较浅的交互表面。

• Each protocol configured on a single router interacts with any other protocols on that same router through shared resources, such as memory and processor. Multiple protocols installed on a single router also share a common RIB (or set of RIBs), so that the removal of a route by one protocol can cause a reaction in the second protocol—perhaps advertising a replacement route, or removing reachability to a destination that was only reachable through the (now removed) destination. The first set of interactions—competition for shared resources—can generally be considered a narrowly scoped shallow interaction surface. The second set of interactions—shared and interactive reachability information through a shared RIB—can be considered a slightly broader, but still shallow, interaction surface.

• 在网络中的每个路由器上配置两种路由协议可拓宽交互面，因为两种协议共享处理器和内存以及通过共享可达性信息进行交互的情况更多。这看起来复杂性并没有大幅增加，但随着网络中的每个路由器都运行这两种协议，任一协议中的单一故障导致大规模中断的机会就会增加。

• Configuring both routing protocols on every router in the network broadens the interaction surface, as there are more instances where the two protocols share processor and memory, as well as interacting through shared reachability information. This might not appear to be a large increase in complexity, but with every router in the network running both protocols, the opportunities for a single failure in either protocol to cause a large-scale outage is increased.

• 重新分发两个协议以在网络中的某一点（在一台路由器上）重新分发可达性信息，会增加该路由器上的交互深度，从而稍微增加复杂性。

• Redistributing the two protocols to redistribute reachability information at one point in the network (on one router) increases the depth of interaction on that one router, increasing the complexity by some small amount.

• 重新分配两个协议以重新分配网络中每个路由器上的可达性信息，增加了交互表面整个广度上的交互深度。这意味着两个协议之间的交互表面的复杂性大大增加。

• Redistributing the two protocols to redistribute reachability information on every router in the network increases the depth of interaction across the entire breadth of the interaction surface. This represents a large increase in the amount of complexity through the interaction surface between the two protocols.

路由协议相互依赖或交互越多，交互面就越深。路由协议交互的地方越多，交互面就越广。

The more the routing protocols rely on one another, or interact, the deeper the interaction surface. The more places the routing protocols interact, the broader the interaction surface.

图2.9说明了交互表面涉及的第三个基本概念：重叠交互。

Figure 2.9 illustrates the third basic concept involved in interaction surfaces: overlapping interactions.

图 2.9 重叠交互

Figure 2.9 Overlapping Interactions

在图 2.9中，组 A 说明了两个重叠或交互的组件或系统，而组 B 说明了三个。随着越来越多的组件或系统在一组接口中进行交互，整个系统变得更加复杂。网络工程中的一个例子是发送数据包的设备与处理数据包通过网络时的设备之间的交互。

In Figure 2.9, set A illustrates two components or systems that overlap, or interact, while set B illustrates three. As more components or systems interact in a single set of interfaces, the overall system becomes more complex. An example of this in network engineering is, again, the interaction between devices that send packets and devices that process packets as they pass through the network.

• 如果数据包只是根据目标地址转发，则路径上的每个路由器实际上都在与发送和接收主机进行交互，但只是以非常浅的方式进行交互。因此，交互重叠度很高，但交互深度很浅。交互的广度取决于数据包从源到目的地必须经过的跳数。

• If a packet is simply forwarded based on the destination address, then each of the routers along the path is actually interacting with the sending and receiving hosts, but only in a very shallow way. Hence, the interaction overlap is high, but the depth of interaction is very shallow. The breadth of interaction would depend on the number of hops through which the packet must pass to travel from the source to the destination.

• 如果一个数据包由一台需要维护数据包之间的状态以及返回流量的设备进行检查，则发送方、接收方和控制平面之间的交互深度相当高（因为任何数据包格式更改或任何网络路径更改都会影响数据包格式）。需要由状态数据包检测设备考虑）。但如果网络中只有一个地方发生这种交互，那么就只有三个系统相交。

• If a packet is inspected by one device that requires state maintained between packets, and about return traffic, the interaction depth is fairly high between the sender, receiver, and control plane (as any packet format changes, or any network path changes, will need to be accounted for by the stateful packet inspection device). But if there is only one place in the network where this interaction is taking place, then there are only the three systems intersecting.

• 网络中添加状态数据包检测的每个点都会增加交互系统的数量，从而增加交互的重叠。

• Each point at which stateful packet inspection is added in the network increases the number of systems interacting, and hence the overlapping interactions.

重叠的组件或系统越多，整个系统就越复杂。

The more overlapping components or systems, the more complex the overall system is.

沙漏模型

The Hourglass Model

为了在现实世界条件下提供潜在的稳健性，复杂性是必要的——重复奥尔德森和多伊尔在第一章“定义复杂性”中首次遇到的陈述：

Complexity is necessary to provide the underlying robustness in real-world conditions—to repeat a statement by Alderson and Doyle first encountered in Chapter 1, “Defining Complexity”:

具体来说，我们认为，高度组织化的系统的复杂性主要源于旨在为其环境和组件的不确定性创造稳健性的设计策略。⁵

Specifically, we argue that complexity in highly organized systems arises primarily from design strategies intended to create robustness to uncertainty in their environments and component parts.⁵

5 . David L. Alderson 和 John C. Doyle，“复杂性的对比观点及其对以网络为中心的基础设施的影响”，IEEE Transactions on Systems, Man, and Cybernetics 40，no。4（2010 年 7 月）：840。

5. David L. Alderson and John C. Doyle, “Contrasting Views of Complexity and Their Implications for Network-Centric Infrastructures,” IEEE Transactions on Systems, Man, and Cybernetics 40, no. 4 (July 2010): 840.

工程师面临着管理复杂性的问题。有一个简单的模型在自然界中普遍存在，并且在工程界被广泛模仿。虽然工程师并不经常有意识地应用这个模型，但它实际上一直在使用。这个型号是什么？图 2.10说明了沙漏模型。

Engineers are left with the problem of managing complexity. There is a simple model that is ubiquitous throughout the natural world, and is widely mimicked in the engineering world. While engineers don’t often consciously apply this model, it’s actually used all the time. What is this model? Figure 2.10 illustrates the hourglass model.

图 2.10 沙漏模型

Figure 2.10 The Hourglass Model

例如，考虑网络世界中广泛使用的四层模型。图 2.11将常用的网络协议四层模型与图 2.10所示的沙漏模型进行了比较。

As an example, consider the four-layer model used so widely in the networking world. Figure 2.11 compares the commonly used four-layer model for network protocols to the hourglass model shown in Figure 2.10.

图 2.11 沙漏模型与四层网络模型的比较

Figure 2.11 The Hourglass Model Compared to the Four-Layer Network Model

在底层，即物理传输系统，有各种各样的协议，从以太网到卫星。在顶层，信息被编组并呈现给应用程序，有各种各样的协议，从 HTTP 到 TELNET（以及数千种其他协议）。然而，当您向堆栈中间移动时，会发生一件有趣的事情：协议数量减少，形成沙漏。为什么这可以控制复杂性？回顾复杂性的三个组成部分——状态、速度和表面——揭示了沙漏和复杂性之间的关系。

At the bottom layer, the physical transport system, there are a wide array of protocols, from Ethernet to Satellite. At the top layer, where information is marshalled and presented to applications, there are a wide array of protocols, from HTTP to TELNET (and thousands of others besides). However, a funny thing happens when you move toward the middle of the stack: the number of protocols decreases, creating an hourglass. Why does this work to control complexity? Going back through the three components of complexity—state, speed, and surface—exposes the relationship between the hourglass and complexity.

•状态按沙漏分为两种不同类型的状态：有关网络的信息和有关通过网络传输的数据的信息。上层关注以可用的方式编组和呈现信息，而下层关注发现存在什么连接以及该连接的实际属性是什么。下层不需要知道如何格式化 FTP 帧，上层也不需要知道如何通过以太网传输数据包——模型两端的状态都会减少。

• State is divided by the hourglass into two distinct types of state: information about the network, and information about the data being transported across the network. While the upper layers are concerned with marshalling and presenting information in a usable way, the lower layers are concerned with discovering what connectivity exists and what the properties of that connectivity actually are. The lower layers don’t need to know how to format an FTP frame, and the upper layers don’t need to know how to carry a packet over Ethernet—state is reduced at both ends of the model.

•通过隐藏层之间的信息来控制速度。正如相同信息的并行副本可以是“速度倍增器”一样，隐藏信息可以是“减速器”，或者可能是一组制动器。如果可以在一层中处理信息而不涉及另一层的状态，则向任何特定层呈现新状态的速度就会降低。例如，提供给 FTP 客户端的数据中的错误不会导致 TCP 状态发生变化，更不用说以太网链路状态了。

• Speed is controlled by hiding information between layers. Just as parallel copies of the same information can be a “speed multiplier,” hiding information can be a “speed reducer,” or perhaps a set of brakes. If information can be handled at in one layer without involving the state of another layer, then the speed at which a new state is presented to any particular layer is reduced. For instance, an error in the data presented to an FTP client doesn’t cause a change of state in the state of TCP, much less in the state of the Ethernet link.

•通过将各个组件之间的交互点数量减少到精确的一个（IP）来控制表面。可以通过标准流程很好地定义这一单一交互点，并严格监管一个交互点的变化，以防止反映协议栈上下的大规模快速变化。

• Surfaces are controlled by reducing the number of interaction points between the various components to precisely one—IP. This single interaction point can be well defined through a standard process, with changes in the one interaction point closely regulated to prevent massive rapid changes that will reflect up and down the protocol stack.

因此，堆叠网络模型的分层是控制网络各个交互组件的复杂性的直接尝试。

The layering of a stacked network model is, then, a direct attempt to control the complexity of the various interacting components of a network.

七层模型已死

The Seven-Layer Model Is Dead

七层模型在网络工程领域几乎被普遍教授和使用——但随着时间的推移，它的实用性一直在稳步下降。有两个不同的问题困扰着七层模型的实用性。首先，该模型越来越受到在协议之上叠加协议的趋势的困扰，并通过通过更窄的通道推送越来越多的信息来避免与中间盒的交互。在 IPv6 上运行的 QUIC 在以太网上运行在虚拟可扩展局域网 (VXLAN) 上运行在 IPv4 上运行充其量很难描述使用七层模型；以太网是在模型中的 VXLAN 隧道第 2 层还是第 4 层之上运行？是否应该将多个七层模型叠加起来来描述这种情况？多协议标签交换 (MPLS) 是第 2 层协议还是第 3 层协议？这是隧道还是不是隧道？在七层模型的背景下，这些问题即使不是不可能，也很难回答。其次，为了避免与中间盒进行深度交互，许多应用程序只是在已知对大多数安全设备和/或服务“开放”的端口上运行。例如，大量通过 Internet 的流量都是通过 HTTP 协议使用 TCP 端口 80 传递的，即使该应用程序可能不是网站。

The seven-layer model is taught and used almost universally within the network engineering world——but its usefulness has been steadily decreasing over time. Two different problems plague the usefulness of the seven-layer model. First, the model is increasingly beset by the tendency to overlay protocol on top of protocol, and to avoid interaction with middle boxes by pushing ever greater amounts of information through a narrower channel. QUIC running over IPv6 running over Ethernet running over Virtual Extensible Local Area Network (VXLAN) running over IPv4 is difficult, at best, to describe using the seven-layer model; is Ethernet running on top of the VXLAN tunnel layer 2 in the model, or Layer 4? Should multiple seven-layer models be laid on top of one another to describe this situation? Is Multiprotocol Label Switching (MPLS) a Layer 2 protocol, or a Layer 3 protocol? Is it a tunnel, or not? These questions are difficult, if not impossible, to answer within the context of the seven-layer model. Second, to avoid deep interactions with middle boxes, many applications simply run over ports known to be “open” for most security devices and/or services. For instance, a large amount of the traffic crossing the Internet is passed through the HTTP protocol, using TCP port 80, even though the application may not be a website.

这两个问题都可以追溯到两个根源。首先，七层模型过度指定了数据传输问题空间。七层模型并没有纯粹关注功能，而是尝试引入交互点和局部性。第 1 层不仅是一组功能，也是一个位置——单个物理链路。第 3 层不仅是一组功能，而且还是一个位置——端到端（理论上）。其次，七层模型实际上是为一组特定的传输协议而设计的——这些协议已不再广泛使用。相反，七层模型用于描述四层协议栈，即 TCP/IP 协议套件。

Both of these problems can be traced back to two sources. First, the seven-layer model overspecifies the data transport problem space. Rather than focusing purely on functionality, the seven-layer model also tries to carry in interaction points and locality. Layer 1 is not only a set of functionality, but also a location—a single physical link. Layer 3 is not only a set of functionality, but also a location—end-to-end (in theory). Second, the seven-layer model was really designed for a specific set of transport protocols—protocols that are no longer in wide use. Instead, the seven-layer model is used to describe a four-layer protocol stack, the TCP/IP suite of protocols.

那么工程师是否应该放弃网络模型呢？不，更好的想法是重新考虑工程师用来描述网络的模型集。与其专注于以主机或网络设备为中心的网络传输系统视图，不如使用多个模型来分解系统。一种模型可用于描述传输，另一种用于描述网络设备，第三种用于描述主机与网络的交互，第四种用于描述用于在整个网络中提供可达性信息的各种类型的控制平面。对于传输模型，更好的重点是提供一组服务所需的功能，并在每个位置使用迭代层。这将更能描述解决每种情况下的具体问题所面临的问题。

Should engineers simply abandon network models, then? No—a better idea would be to rethink the set of models engineers use to describe the network. Rather than focusing on a host, or network device, centric view of the network transport system, it is better to break the system up using multiple models. One model can be used to describe transport, another to describe network devices, a third used to describe a host’s interaction with the network, and a fourth used to describe the various types of control planes used to provide reachability information throughout the network. For transport models, a better focus would be the functions needed to provide a set of services, with iterative layers used at each locality. This would be more descriptive of the problems being brought to bear to solve specific problems in each case.

有关网络模型的更多信息，请参阅Cisco Press 标题《网络架构的艺术》中的第 4 章“模型” 。

For more information on network models, see Chapter 4, “Models,” in the Cisco Press title, The Art of Network Architecture.

优化

Optimization

虽然本书将使用状态、速度和表面来描述复杂性，但工程师通常需要考虑第四个组件——优化。很多时候，复杂性是对优化的权衡。增加复杂性会增加网络的优化，降低复杂性会降低网络的优化。这一原理的例证可以在检查事件驱动反应和定时器驱动反应对网络变化的反应之间的选择中找到。图 2.12说明了定时器和事件驱动的检测。

While state, speed, and surface will be used to describe complexity throughout this book, there is a fourth component engineers often need to take into account—optimization. Quite often, complexity is a tradeoff against optimization; increasing complexity increases the optimization of the network, and reducing complexity reduces the optimization of the network. An illustration of this principle can be found in examining the choice between event-driven reactions and timer-driven reactions to changes in the network. Figure 2.12 illustrates timer- and event-driven detection.

图 2.12 定时器和事件驱动的检测

Figure 2.12 Timer- and Event-Driven Detection

在图2.12中，从左到右显示了时间线。随着时间的推移，两个 OSPF 进程会定期交换 hello 数据包——这是定时器驱动检测系统的典型示例。如果两个 OSPF 进程之间的链路发生故障，两个进程将通过丢失三个 hello 数据包来识别故障，从而导致邻接失败，并且通过丢失的邻居获知的路由将从本地数据库和路由表中删除。第二个检测过程也通过链路载体、接口驱动程序、路由表并进入 OSPF 过程 1 来显示。这是一个事件驱动的检测链：

In Figure 2.12, a timeline is shown from left to right. Over time, two OSPF processes are exchanging periodic hello packets—a classic example of a timer-driven detection system. If the link between the two OSPF processes fails, the two processes will recognize the failure through a loss of three hello packets, causing the adjacency to fail and the routes learned through the lost neighbor to be removed from the local database and routing table. A second detection process is also shown through the link carrier, interface driver, routing table, and into OSPF Process 1. This is an event-driven detection chain:

• 如果链路出现故障，物理接口上的载波检测将会失败。这将导致物理接口向接口驱动程序发出已发生故障的信号。

• If the link fails, carrier detection on the physical interface will fail. This will cause the physical interface to signal the interface driver that the failure has occurred.

• 当接口驱动程序收到通知后，它将向路由子系统发出信号，路由子系统将从任何受影响的 RIB 中删除通过现在发生故障的接口可到达的任何路由。

• When the interface driver is notified, it will then signal the routing subsystem, which will remove any routes reachable through the now failed interface from any impacted RIB.

• 当RIB 删除受影响的路由（包括连接的接口路由）时，它将向OSPF 进程1 发出故障信号。这将导致 OSPF 进程从其本地表中删除通过该接口可到达的任何邻居，以及从其本地表中从该邻居获悉的任何链路状态数据库条目。

• When the RIB removes the effected routes, including the connected interface route, it will signal OSPF Process 1 of the failure. This will cause the OSPF process to remove any neighbors reachable through that interface from its local tables, and any link state database entries learned from this neighbor from its local tables.

在此示例中，事件驱动的检测更为复杂，因为事件必须通过多个接口才能到达 OSPF 进程 1。这些接口中的每一个都意味着必须管理一个交互表面；事实上，这个交互表面可能很深，因为 OSPF 进程可能需要根据链路类型、故障类型或较低层提供的其他信息做出不同的反应。事件驱动的检测还提高了控制平面状态变化的速度；每个链路抖动都可以单独记录在链路上运行的 OSPF 进程中，并且这些抖动很可能以拓扑更新的形式在整个控制平面中传输。基于定时器的系统要简单得多；OSPF 进程对用于传输 hello 数据包的底层网络了解不多，

Event-driven detection is more complex in this example, as the event must pass through multiple interfaces to reach OSPF Process 1. Each of these interfaces implies an interaction surface that must be managed; this interaction surface may, in fact, be deep, as the OSPF process may need to react differently depending on the link type, the type of failure, or other information provided from the lower layers. Event-driven detection also increases the speed of state change in the control plane; each link flap may be individually recorded in the OSPF process running over the link, and these flaps could well be transmitted throughout the control plane in the form of topology updates. The timer-based system is much simpler; the OSPF processes don’t have a lot of knowledge about the underlying network being used to transport the hello packets, and the state of the adjacency is changed only at fixed intervals, dampening any potential feedback loops, and slowing down the rate of change in the control plane (speed).

在这个例子中，优化权衡也应该很清楚。事件驱动的检测过程将更快地发现下行链路，从而使控制平面能够非常快速地通过链路路由来对故障做出反应。这减少了平均修复时间 (MTTR)，从而提高了整体网络可用性。在这种情况下，更复杂的事件驱动过程会增加优化，而选择降低复杂性也会导致网络优化的减少。

The optimization tradeoff should be clear in this example, as well. The event-driven detection process will discover the downlink much faster, allowing the control plane to react to the failure by routing around the link—for instance—very quickly. This reduces the Mean Time to Repair (MTTR), and hence increases the overall network availability. In this case, then, the more complex event-driven process increases optimization, while opting for reduced complexity also incurs a reduction in network optimization.

增加优化并不总是需要增加复杂性，或者尝试降低复杂性总是会降低优化，但这种情况很常见。这种权衡的例子遍布本书各处。

It isn’t always going to be the case that increasing optimization will require increasing complexity, or attempts to reduce complexity will always decrease optimization—but it is quite common. Examples of this tradeoff are scattered throughout this book.

最后的想法

A Final Thought

状态、速度、表面以及优化——如果您能够围绕这四个组成部分进行思考，您就可以牢牢掌握网络复杂性所涉及的问题。从未真正融合的网络正在成为常态而不是例外；传统模式正在崩溃。如果工程师能够学习如何识别复杂性并在网络工程难题的各个方面对其进行管理，那么沙漏模型就提供了一条穿越复杂性泥沼的前进之路。

State, speed, and surface, and optimization—if you can get your thinking around these four components, you can get a solid grip on the problems involved in network complexity. Networks that never truly converge are becoming the norm rather than the exception; the traditional models are breaking down. The hourglass model provides a way forward through the complexity morass, if engineers can learn how to recognize complexity and manage it in all the various pieces of the network engineering puzzle.

第 3 章测量网络复杂性

Chapter 3. Measuring Network Complexity

考虑到复杂性的这四个基本方面——状态、速度、表面和优化——只有测量这四个点并生成一个描述给定设计和部署结构的整体复杂性的数字才有意义。如果有某种方法可以检查提议的网络设计或对网络设计的提议更改，并且能够为每个组件的复杂性分配实际数字，以便可以将复杂性与任何潜在的性能增益进行比较，那就太好了，或者可以将一个区域的复杂性损失与另一区域的复杂性增益进行比较。如果事情就这么简单就好了。

Given these four fundamental aspects of complexity—state, speed, surface, and optimization—it only makes sense to measure these four points and generate a single number describing the overall complexity of a given design and deployment structure. It would be nice if there were some way to examine a proposed network design, or a proposed change to a network design, and be able to assign actual numbers to the complexity of each component so the complexity can be compared to any potential gain in performance, or the loss of complexity in one area can be compared to the gain in complexity in another. If it were only that simple.

事实证明，衡量复杂性的工作本身就相当复杂。

As it turns out, the effort to measure complexity is, itself, quite complex.

在检查测量和量化网络的问题以了解整个系统的复杂性时，两个问题浮出水面。首先，可用信息量巨大。鉴于当前对大数据分析的推动，以及测量数千到数百万次交互和数据挖掘以发现重要趋势和工件的能力，平均网络规模的问题难道不是一个简单的问题吗？考虑一些不同的测量点，只是为了了解流经网络中每个点的数据与用于处理该流量的排队机制之间的交互。这可能包括以下内容：

Two problems rise to the surface when examining the problem of measuring and quantifying a network toward gaining an understanding of the overall system complexity. First, there is the sheer amount of information available. Given the current push toward big data analytics, and the ability to measure thousands to millions of interactions and data mining to discover important trends and artifacts, shouldn’t something the size of an average network be an easy problem? Consider some of the various points of measurement just in trying to understand the interaction between the data flowing through each point in the network and the queuing mechanisms used to handle that traffic. This might include things such as:

• 流经网络中每个点的数据量，包括每个转发设备的输入和输出队列。

• The amount of data flowing through each point in the network, including the input and output queue of each forwarding device.

• 网络中每个转发设备的每个队列的深度和状态。

• The depth and state of each queue of each forwarding device in the network.

• 通过网络转发的每个数据包的源、目的地和其他标头信息。

• The source, destination, and other header information of each packet forwarded through the network.

• 每个转发设备丢弃的数据包数量，包括丢弃的原因（尾部丢弃、数据包错误、已过滤、过滤规则等）。

• The number of packets dropped by each forwarding device, including the reason why they were dropped (tail drop, packet error, filtered, filtering rule, etc.).

考虑到测量本身必须通过网络，并且测量很容易包含比测量流量更多的流量，测量所有内容的问题很快就会变得明显。如果测量与您正在测量的内容在同一通道上进行，如何将测量与被测量分开？除此之外，每个控制平面系统的状态以及这些系统的组件也增加了这一挑战，例如每个转发设备的内存和处理器利用率、参与控制平面的每对设备之间的每个邻接状态，以及控制平面内每个可达性广告的流程。为了使测量系统复杂性变得更加复杂，系统之间的交互也必须以某种方式考虑在内——诸如可达性信息对策略的分发和应用的影响、并行控制平面之间在可达性信息和系统资源方面的任何相互依赖性，以及覆盖层和底层控制之间的交互飞机。不仅测量系统，而且测量系统之间的交互很快就成为一个棘手的问题。

Considering that the measurements themselves must pass through the network—and the measurements can easily contain more traffic than the measured traffic—the problems with measuring everything should quickly become apparent. How can you separate the measurement from the measured if the measurement is being carried on the same channel as what you are measuring? Added to this challenge are the states of each individual control plane system, and the components of those systems—things like the memory and processor utilization of each forwarding device, the state of each adjacency between each pair of devices participating in the control plane, and the flow of each reachability advertisement within the control plane. To make measuring the system complexity even more complex, the interactions between the systems must also somehow be taken into account—things like the impact of reachability information on the distribution and application of policy, any interdependencies between parallel control planes in terms of reachability information and system resources, and interactions between overlay and underlay control planes. Measuring not only the systems but also the interactions between the systems quickly becomes an intractable problem.

当测量一个系统以了解其复杂程度时，必须进行某种采样。采样必然意味着必须排除一些信息，这反过来又意味着任何对复杂性的测量都必然是复杂性的抽象表示，而不是对复杂性本身的测量。

When measuring a system to understand its complexity level, some sort of sampling must take place. Sampling necessarily means that some information must be left out—which, in turn, means that any measurement of complexity along these lines is necessarily an abstract representation of the complexity, rather than a measure of the complexity itself.

最重要的是，对于创建网络复杂性的准确抽象表示所需测量的一组事物，几乎没有达成一致。

To top all of this complexity off, there is very little agreement on the set of things to measure to create even an accurate abstract representation of the complexity of a network.

继第一个问题之后，第二个问题迫在眉睫——这个问题并不那么明显，实际上使测量网络复杂性的问题变得棘手。网络设计代表有序（或有意或有组织——这三个术语经常互换使用）的复杂性，而不是无序的复杂性。虽然数据分析可以很好地处理无序数据，但有序复杂性是一个完全不同的问题集。

There is a second problem looming on the horizon just past this first one—a problem that’s not so obvious, and actually makes the problem of measuring network complexity intractable. Network design represents ordered (or intentional or organized—these three terms are often used interchangeably) complexity, rather than unordered complexity. While data analytics deals with unordered data well enough, ordered complexity is an entirely different problem set.

让我们首先检查一些用于测量网络复杂性的方法，然后考虑有序复杂性与无序复杂性。最后，将研究几个复杂的领域，这些领域将导致实际应用。

Let’s begin by examining some methods proposed to measure network complexity, and then consider ordered versus unordered complexity. Finally, several realms of complexity will be examined that will lead to practical applications.

网络复杂性的一些衡量标准

Some Measures of Network Complexity

这项任务的难度并没有阻止研究人员尝试测量网络复杂性。恰恰相反——多年来人们已经尝试过多种方法。这些方法中的每一种都为问题空间贡献了有用的思考，并且实际上可以用于提供对网络复杂性的一些了解。但总的来说，这些测量都无法真正提供网络复杂性的完整视图。

The difficulty of the task hasn’t stopped researchers from attempting to measure network complexity. Quite the opposite—there are a number of methods that have been tried over the years. Each of these methods has contributed useful thinking to the problem space, and can actually be used to provide some insight into what network complexity looks like. Overall, though, none of these measurements will truly provide a complete view of the complexity of a network.

让我们看一下网络复杂性测量的三个示例，以了解一下这个空间。

Let’s look at three examples of network complexity measurements to get a feel for the space.

网络复杂度指数

Network Complexity Index

Bailey 和 Grossman（通常称为 NCI）在“网络的网络的网络复杂性指数” ^{1中描述了网络复杂性指数。}总体思路是分两步解决描述网络复杂性的问题：

The Network Complexity Index is described in “A Network Complexity Index for Networks of Networks”¹ by Bailey and Grossman (commonly called the NCI). The general idea is to tackle describing network complexity in two steps:

1 . Stewart Bailey 和 Robert L. Grossman，“网络网络的网络复杂性指数”（Infoblox，2013 年），np，https: //web.archive.org/web/20131001093751/http ://flowforwarding.org/docs/ Bailey%20-%20Grossman%20article%20on%20network%20complexity.pdf。

1. Stewart Bailey and Robert L. Grossman, “A Network Complexity Index for Networks of Networks” (Infoblox, 2013), n.p., https://web.archive.org/web/20131001093751/http://flowforwarding.org/docs/Bailey%20-%20Grossman%20article%20on%20network%20complexity.pdf.

• 将网络分解为子网。正如原论文中所述：

• Break the network down into subnetworks. As described in the original paper:

给定一个网络 N，我们首先将网络划分为更小的子网络 C[1],…。。., C[j], . 。。, C[p] 具有以下属性：从子网 C[i] 中随机选择的两个节点比从子网外部随机选择的两个节点 (N\C) 更有可能相互连接。

Given a network N, we first divide the network into smaller sub-networks C[1], . . ., C[j], . . . , C[p] with the property that two nodes selected at random from the sub-network C[i] are more likely to be connected to each other than two nodes selected at random from outside the sub-network (N\C).

• 根据子网的大小和数量计算复杂性。再次，如原始论文中所述：

• Compute the complexity based on the size and number of the subnetworks. Again, as described in the original paper:

给定网络 N 的子社区，令 X[j] 表示第 j 个最大子社区的大小，使得序列 X[1],... 。。, X[p] 按降序排列。一般来说，不同的社区可能具有相同的规模。我们将网络 N 的网络复杂度指数 B(N) 定义为以下方程的解： B(N) = Max j, X[j] j

Given the sub-communities of the network N, let X[j] denote the size of the j largest sub-community, so that the sequence X[1], . . . , X[p] is in decreasing order. In general, different communities may have the same size. We define the network complexity index B(N) of the network N as the solution of the following equation: B(N) = Max j, X[j] j

给出的方程是用于评估科学研究重要性的标准统计数据，称为 H 指数。H 指数决定了影响一项特定的研究，通过评估作品的引用次数，类似于网页搜索索引，使用页面的链接数量来确定该页面的重要性或相关性。

The equation given is a standard statistic used in evaluating the importance of scientific research known as the H-index. The H-index determines the impact of a particular piece of research by evaluating the number of citations of the work in a way that is similar to a web page search index using the number of links to a page to determine the importance or relevance of that page.

从这个角度来看，NCI 尝试将网络内的连接性与网络内的节点数量结合起来：

Seen this way, the NCI attempts to combine the connectivity within a network with the number of nodes within a network:

• 子社区越多，这些子社区之间的连接点就越多，因此连接图也就越复杂。

• The more the subcommunities, the more connection points there must be between these subcommunities, and hence the more complex the connection graph must be.

• 子社区越大，网络内的节点就越多；这再次影响网络的隐含连接图

• The larger the subcommunities, the more nodes there are within the network; this again impacts the implied connectivity graph of the network

连接图的大小和范围反过来会影响信息在网络内流动的方式，这也与网络的复杂性有关。

The size and scope of the connectivity graph, in turn, impacts the way information flows within the network, which also relates to the complexity of the network.

NCI 擅长什么

NCI 很好地生成了一个数字，该数字描述了网络在节点和社区方面的规模和形状，进而暗示了网络互连的范围和复杂性。随着时间的推移计算出的这个单一数字可以帮助网络管理员和设计人员了解网络的增长情况，而不仅仅是单纯的规模。

The NCI does a good job of producing a single number that describes the size and shape of a network in terms of nodes and communities, in turn implying the scope and complexity of the network interconnections. This single number, computed over time, can help network managers and designers understand the growth of a network in terms other than sheer size.

NCI 不做什么

从网络工程师的角度来看，使用 NCI 作为网络复杂性的单一衡量标准存在几个实际问题。首先，这不是你在吃晚饭时在餐巾纸上计算的东西，也不是在头脑中进行粗略计算的东西。²这是一项数学繁重的计算，需要自动化工具来计算。其次，除了测量拓扑的增长和互连性之外，很难看出 NCI 在现实世界中如何以及在何处有用。根据 NCI 的测量，除了减少网络中子社区的数量和大小之外，没有明显的方法可以降低网络复杂性。

From a network engineer’s perspective, there are several practical problems with using the NCI as a single measure of network complexity. First, this isn’t something you’re going to compute on a napkin while you’re eating dinner, or do rough calculations in your head around.² This is a math heavy computation that requires automated tools to compute. Second, other than measuring the growth and interconnectedness of a topology, it’s hard to see how and where the NCI is useful in the real world. There’s no obvious way to reduce network complexity as measured by the NCI other than reducing the number and size of the subcommunities in the network.

2 . 事实上，一个名为 Tapestry 的整个项目是围绕通过自动收集配置并通过处理器运行它们来测量 NCI 构建的。该项目可以在 GitHub 上找到：https://github.com/FlowForwarding/tapestry。

2. In fact, an entire project called Tapestry was built around measuring the NCI by gathering configurations automatically and running them through a processor. The project can be found on GitHub at https://github.com/FlowForwarding/tapestry.

然而，第二个反对意见导致了 NCI 的另一个缺点：它并没有真正衡量网络运营商交互的复杂性。是相当在现实世界中，经常会发现非常大的网络仅支持少数工作负载，并且这些网络已针对该工作负载进行了大量优化，因此从工程师的角度来看并不是很复杂。在现实世界中，发现具有非常多样化的工作负载的小型网络也很常见，因此无法针对单个工作负载进行优化。这些网络比其规模所显示的更为复杂——NCI 可能会低估这些网络的复杂性。

This second objection, however, leads to another shortcoming of the NCI: it doesn’t really measure the complexity network operators interact with. It’s quite common, in the real world, to find very large networks supporting only a few workloads that have been heavily optimized for that workload, and hence are not very complex from an engineer’s point of view. It’s also quite common, in the real world, to find small networks with a very diverse workload, and hence cannot be optimized for a single workload. These networks are more complex than their size indicates—the NCI would likely underestimate the complexity of these networks.

那么 NCI 错过了什么？只是设计人员大多数时间处理的网络架构部分，例如：

So what does the NCI miss? Just those pieces of network architecture that designers deal with most of the time, such as:

• 策略，通过配置、指标、协议和其他方法表达

• Policy, expressed through configuration, metrics, protocols, and other methods

• 弹性，通过冗余量、快速收敛机制和其他高度复杂的设计组件来表示

• Resilience, expressed through the amount of redundancy, fast convergence mechanisms, and other highly complex design components

因此，虽然 NCI 很有用，但它并不能以可有效应用于现实世界网络的方式捕获单个网络的所有复杂性。

So while the NCI is useful, it doesn’t capture all the complexity of a single network in a way that can be usefully applied to real-world networks.

建模设计复杂性

Modeling Design Complexity

在向互联网研究任务组的网络复杂性研究小组展示的一组幻灯片中，一组研究人员描述了一个用于测量和描述企业路由设计复杂性的模型。³测量过程如下：

In a set of slides presented to the Internet Research Task Force’s Network Complexity Research Group, a group of researchers described a model for measuring and describing the complexity of enterprise routing design.³ The process of measurement is as follows:

3 . Xin Sun、Sanjay G. Rao 和 G.Xie Geoffrey，“企业路由设计的复杂性建模”（IRTF NCRG，2012 年 11 月 5 日），np，2014 年 10 月 5 日访问，http: //www.ietf.org/诉讼程序/85/slides/slides-85-ncrg-0.pdf。

3. Xin Sun, Sanjay G. Rao, and G.Xie Geoffrey, “Modeling Complexity of Enterprise Routing Design” (IRTF NCRG, November 5, 2012), n.p., accessed October 5, 2014, http://www.ietf.org/proceedings/85/slides/slides-85-ncrg-0.pdf.

1.将网络设计分解为单独的部分，作为单独的配置组件实现（跨网络中的所有设备）。

1. Decompose the network design into individual pieces, implemented as individual configuration components (across all devices in the network).

2.在这些单独的组件之间建立一个连接网络。

2. Build a network of connections between these individual components.

3.测量该网络以确定配置的复杂性。

3. Measure this network to determine the complexity of the configuration.

图 3.1取自这些幻灯片⁴ ，说明了各个配置组件之间的联系。

Figure 3.1 is taken from these slides,⁴ illustrating the linkages between the various configuration components.

4 . 同上。

4. Ibid.

图 3.1 评估网络配置中的链接

Figure 3.1 Evaluating Linkages in a Network Configuration

配置的项目越多，配置之间（特别是盒子之间）的互连越密集，设计就越复杂。通过检查这些因素，该过程会产生一个描述设计复杂性的数字。同一作者撰写的演示文稿和论文扩展了这一概念，以确定所实现的设计是否与设计意图相匹配，前提是设计意图以适合流程的方式表述。

The more items configured, and the more dense the interconnection between the configurations (particularly between boxes), the more complex the design is determined to be. By examining these factors, the process yields a single number describing the complexity of the design. The presentation and the papers written by the same authors extend this concept to determining if the implemented design matches the design intent, given the design intent is stated in a way amenable to the process.

建模设计复杂性有什么好处

不仅测量配置线，而且测量配置线之间的互连，这一概念非常强大。毫无疑问，大部分复杂性来自于遍布整个网络的相互关联的配置，这些配置是实施单一策略或使单一协议或流程工作所需的。查看交互或配置网络，也会将配置行数作为复杂性的衡量标准。由于某些设备可以用几行简单的代码表达策略，而其他设备则需要大量配置来表达相同的策略，因此这是一个有用的结果。

The concept of measuring not just the lines of configuration, but the interconnections between the lines of configuration, is extremely powerful. There is little doubt that much of complexity comes from the interrelated configurations spread throughout a network required to implement a single policy or to make a single protocol or process work. Looking at the interactions, or network of configurations, also takes the number of lines of configuration somewhat out of the picture as a measure of complexity. Because some devices can express a policy in a few simple lines, while others require a lot of configuration to express the same policy, this is a useful outcome.

建模设计复杂性不能做什么

然而，与此同时，配置行之间的互连可能会陷入同样的问题，仅计算配置行的数量就会陷入困境 - 整个策略可能由一台设备上的单行配置表示，而需要制定一系列政策。例如，在 Cisco IOS 软件上，命令remove-private-as用于删除 BGP 路由通告中的任何私有 AS 编号。这单个命令本质上取代了一组必须互连的配置命令，例如过滤器以及该过滤器对一组特定 BGP 对等体的应用。两种配置都是有效的并且执行相同的一组操作，但是根据所描述的度量，它们似乎具有完全不同的复杂性级别。使情况更加复杂的是，不同的 BGP 实现可能使用不同的命令集来执行相同的操作，从而使一个配置看起来比另一个配置更复杂，尽管它们都实现相同的策略。

At the same time, however, the interconnections between lines of configuration can fall prey to the same problems just counting the number of lines of configuration can fall prey to—an entire policy might be represented by a single line of configuration on one device, while requiring a number of lines of policy on another. For instance, on Cisco IOS Software, the command remove-private-as is used to remove any private AS numbers in a BGP route advertisement. This single command essentially replaces a set of configuration commands that would necessarily be interconnected, such as a filter and an application of that filter to a particular set of BGP peers. Both configurations are valid and perform the same set of actions, but they would appear to have completely different complexity levels according to the measure as it’s described. Further complicating the situation, different BGP implementations might use different sets of command to perform the same action, making one configuration appear more complex than another, although they’re both implementing the same policy.

上述措施的另一个失败是，哪些部分组合在一起制定政策并不总是显而易见的。例如，删除 AS 中每个 eBGP 扬声器上的私有 AS 编号的配置在测量中可能看起来不相关；除非您知道配置的意图，否则这些多个配置没有以明显的方式重叠或交互的特定点。因此，某些策略可能很容易被忽略，因为它们由配置组成，没有明显的结合点。

Another failing of the measure described above is that it’s not always obvious what pieces fit together to make a policy. For instance, a configuration removing private AS numbers on every eBGP speaker in an AS might not appear to be related within the measurement; there is no specific point at which these multiple configurations overlap or interact in a way that’s obvious, unless you know the intent of the configuration. Thus some policies might easily be missed as they consist of configurations with no obvious point at which they tie together.

最后，很难评估如何在这种网络复杂性衡量标准中管理用于实施多个策略的单一配置，然而，从复杂性的角度来看，这是最棘手的管理问题之一，因为这正是那些困难的问题之一。管理原本不相关的策略实施之间的交互面。衡量的各种政策如何相互作用？在这一点上，建模设计的复杂性是沉默的。

Finally, it’s difficult to assess how a single configuration used to implement multiple policies would be managed in this measure of network complexity—and yet, this is one of the thorniest problems to manage from a complexity standpoint, as this is precisely one of those difficult to manage interaction surfaces between otherwise unrelated policy implementations. How do the various policies measured interact? On this point, modeling design complexity is silent.

网络综合体

NetComplex

如前所述，NCI 根据规模和感知的子组件来衡量复杂性；建模设计评估互连配置线网络的复杂性。如何根据保持代表网络拓扑的分布式数据库在参与控制平面的所有设备之间同步所需的工作量来衡量复杂性？这正是 NetComplex 所做的。正如 Chun、Ratnasamy 和 Kohler 所说：

As previously discussed, the NCI measures complexity based on scale and perceived subcomponents; modeling design rates complexity on the network of interconnected lines of configuration. What about measuring complexity based on the amount of work needed to keep the distributed database that represents the network topology synchronized across all the devices participating in the control plane? This is precisely what NetComplex does. As Chun, Ratnasamy, and Kohler state:

我们推测，网络系统特有的复杂性源于需要确保状态与其分布式依赖项保持同步。我们在本文中开发的度量标准反映了这一观点，并且我们说明了几种系统，对于这些系统，这种以依赖关系为中心的方法似乎适当地反映了系统的复杂性。⁵

We conjecture that the complexity particular to networked systems arises from the need to ensure state is kept in sync with its distributed dependencies. The metric we develop in this paper reflects this viewpoint and we illustrate several systems for which this dependency centric approach appears to appropriately reflect system complexity.⁵

5 . Byung-Gon Chun、Sylvia Ratnasamy 和 Eddie Kohler，“NetComplex：网络系统设计的复杂性度量”（第五届 Usenix 网络系统设计和实现 NSDI 研讨会，2008 年，2008 年 4 月），np，2014 年 10 月 5 日访问，http ： //berkeley.intel-research.net/sylvia/netcomp.pdf。

5. Byung-Gon Chun, Sylvia Ratnasamy, and Eddie Kohler, “NetComplex: A Complexity Metric for Networked System Designs” (5th Usenix Symposium on Networked Systems Design and Implementation NSDI 2008, April 2008), n.p., accessed October 5, 2014, http://berkeley.intel-research.net/sylvia/netcomp.pdf.

NetComplex 评估网络中的依赖状态链，为每个依赖项分配一个度量，然后根据这些分配的度量计算单个复杂性度量。图 3.2说明了 NetComplex 中的依赖性和复杂性。

NetComplex evaluates the chain of dependent states in the network, assigns a metric to each dependency, and then calculates a single complexity measure based on these assigned metrics. Figure 3.2 illustrates dependency and complexity in NetComplex.

图 3.2 NetComplex 中的依赖性和复杂性

Figure 3.2 Dependency and Complexity in NetComplex

在此图中：

In this figure:

• 路由器 C 和 D 依靠 E 来获取路由器 E 之外的网络的正确视图。

• Routers C and D depend on E to obtain a correct view of the network beyond Router E.

• 路由器 B 依靠路由器 C、D 和 E 来获取路由器 E 之外的网络的正确视图。

• Router B depends on Routers C, D, and E to obtain a correct view of the network beyond Router E.

• 路由器 A 依靠路由器 B、C、D 和 E 来获取路由器 E 之外的网络的正确视图。

• Router A depends on Routers B, C, D, and E to obtain a correct view of the network beyond Router E.

因此，路由器 A“积累”了整个网络中源自 E 之外的信息同步的复杂性。通过这些依赖性，路由器 A 被称为链接到网络中的其余路由器。通过检查这些链接，并将它们与本地状态相结合，保持整个控制平面同步的复杂性可以给出一个单一的度量标准。

Hence, Router A “accumulates” the complexity of synchronization of information originating beyond E through the entire network. Through these dependencies, Router A is said to be linked to the remaining routers in the network. By examining these links, and combining them with the local state, the complexity of keeping the entire control plane synchronized can be given a single metric.

NetComplex 擅长什么

通过关注状态量和状态通过网络传输的方式，NetComplex 很好地描述了控制平面的复杂性。基于此，NetComplex 可用于确定通过网络携带源路由信息所需的额外复杂性，并基于此源路由信息进行转发。Netcomplex 的另一个有用之处是对每个流（而不是每个目的地/虚拟拓扑）转发流量所需的附加状态信息进行度量。

By focusing on the amount of state and the way state is carried through the network, NetComplex does a good job of describing the complexity of a control plane. Based on this, NetComplex is useful for determining the additional complexity required to carry source routing information through the network, and forward based on this source routing information. Another place where Netcomplex would be useful is in putting a metric on the additional state information required to forward traffic on a per flow, rather than per destination/virtual topology basis.

NetComplex 不做什么

然而，NetComplex 专注于单个管理或故障域内的控制平面。例如，无法解释通过路由聚合隐藏的信息，也无法区分拓扑信息和可达性（例如在链路状态泛洪域边界处发生的情况）。NetComplex 不与策略一起工作，也不与策略实施一起工作；它也不涉及流量、子网规模或网络密度。

NetComplex, however, is focused on the control plane within a single administrative or failure domain. There is no way, for instance, to account for the information hidden through route aggregation, nor to differentiate between topology information and reachability (such as what happens at a link state flooding domain boundary). NetComplex doesn’t work with policies, nor policy implementation; nor does it deal with traffic flows, subnetwork scale, or network density.

有组织的复杂性

Organized Complexity

目前已经研究了网络复杂性的三种不同衡量标准：NCI、建模设计复杂性和 NetComplex。这些方法中的每一个都试图以某种方式至少测量网络复杂性的四个领域（状态、速度、表面和优化）的某些组成部分。然而，它们都没有衡量这三个领域中任何一个领域的所有内容，甚至没有一个能够接近衡量整体网络复杂性。为什么？问题不仅仅是测量和处理产生单个复杂性数字所需的所有信息的能力，它还嵌入到网络复杂性本身的问题中。

Three different measures of network complexity have been examined at this point: NCI, modeling design complexity, and NetComplex. Each of these attempts to measure, in some way, at least some component of the four realms of network complexity—state, speed, surface, and optimization. None of them, however, measure everything in any one of these three domains, and none of them even come close to measuring overall network complexity. Why? The problem isn’t just the ability to measure and process all the information needed to produce a single complexity number, it’s embedded in the problem of network complexity itself.

想象一下，一张台球桌上放着一组球。这些特定的球的弹性（至少接近）完美，因此当它们撞击另一个物体时，它们只会损失无限少量的能量，并且桌子两侧的保险杠的设计方式大致相同。这张桌子上也没有口袋，所以球没有地方离开桌子。现在，将球以某种随机分布的方式放在桌子上，然后敲击其中一个，从而引发连锁反应。结果将是一组统计上随机的运动，每个球在桌子上移动，撞击另一个球或保险杠，保留其大部分能量，然后沿直线向其他方向移动。

Imagine, for a moment, a pool table with a set of balls on it. These specific balls are (at least nearly) perfect in their resilience, so they lose only infinitely small amounts of energy when they strike another object, and the bumpers on the sides of the table are designed in much the same way. There are no pockets in this table, either, so there is no place for the balls to leave the table. Now, place the balls on the table in some random distribution, and then strike one so it starts a chain reaction. The result will be a statistically random set of movements, each ball moving about the table, striking another ball or a bumper, retaining most of its energy, and then moving in a straight line in some other direction.

这个特殊问题对于统计回归分析或数据科学可以提供的任何其他形式的分析来说已经成熟。数据科学家可以根据一组推导公式告诉您，一个球撞击另一个球的频率、系统需要多长时间才能耗尽能量、随机移动的球在什么时间会形成什么模式，以及许多信息。其他事情。数据科学擅长在看似随机的数据中寻找模式。事实上，我们经常发现数据集必须更大才能做出准确的预测；数据集越大，对其进行表征就越准确，并且对该数据在未来某个特定点的状态的预测也就越准确。

This particular problem is ripe for statistical regression analysis, or any other form of analysis data science can provide. The data scientist can tell you, based on a set of derived formulas, how often one ball will strike another, how long the system will take to run out of energy, what patterns will form in the randomly moving balls at what time—and many other things. Data science excels at finding patterns in seemingly random bits of data. In fact, it is often found that the data set must be larger to make an accurate prediction; the larger the data set, the more accurate characterization of it can be made, and the more accurate the predictions about the state of that data at some specific point in the future will be.

但让我们稍微改变一下情况。让我们以相同的台球桌、相同的球——所有相同的身体条件为例。只是这一次，有人预先计划了每个球的位置和运动，这样即使两个球都在运动，也不会互相撞击。事实上，每个球的运动在整个运动过程中都是相同的。

But let’s change the situation somewhat. Let’s take the same pool table, the same balls—all the same physical conditions. Only this time, someone has preplanned the position and movement of every ball such that no two balls strike one another, even though they are all in motion. In fact, the movement of every ball is identical throughout the entire time the balls are in motion.

关于这种特殊情况，数据科学可以告诉我们什么？什么都没有。

What can data science tell us about this particular situation? Nothing.

简单的观察可以告诉我们哪个球在任何时间点都会在哪里。简单的观察甚至可以提供一个公式，告诉我们桌子上哪里会出现球团，或者险些脱手。但统计分析不能超出一些简单的事实。更有趣的是，统计分析无法告诉我们以这种方式排列这些球有什么意义。

Simple observation can tell us which ball will be where at any point in time. Simple observation might even be able to provide a formula telling us where there will be clumps of balls on the table, or near misses. But statistical analysis cannot go much beyond a few simple facts here. What’s more interesting is that statistical analysis cannot tell us what the point is in having these balls arranged just this way.

这就是组织复杂性的问题。

This is the problem of organized complexity.

正如 Warren Weaver 在 1948 年指出的那样：

As Warren Weaver noted in 1948:

这种处理无序复杂性的新方法比早期的二变量方法有了如此强大的进步，留下了一个广阔的领域。人们很容易过于简单化，并说科学方法从一个极端走向另一个极端——从两个变量到一个天文数字——而没有触及一个伟大的中间区域。此外，这个中间区域的重要性并不主要取决于所涉及的变量数量是否适中——与两个相比较大，但与一小撮盐中的原子数量相比却很小。事实上，这个中间区域的问题往往会涉及相当多的变数。这个中部地区问题的真正重要特征，科学迄今尚未探索或征服，在于这些问题，与统计可以应对的无组织情况相比，它显示了组织的基本特征。事实上，我们可以将这组问题称为有组织的复杂性问题。⁶

This new method of dealing with disorganized complexity, so powerful an advance over the earlier two-variable methods, leaves a great field untouched. One is tempted to oversimplify, and say that scientific methodology went from one extreme to the other—from two variables to an astronomical number—and left untouched a great middle region. The importance of this middle region, moreover, does not depend primarily on the fact that the number of variables involved is moderate—large compared to two, but small compared to the number of atoms in a pinch of salt. The problems in this middle region, in fact, will often involve a considerable number of variables. The really important characteristic of the problems of this middle region, which science has as yet little explored or conquered, lies in the fact that these problems, as contrasted with the disorganized situations with which statistics can cope, show the essential feature of organization. In fact, one can refer to this group of problems as those of organized complexity.⁶

6 . 沃伦·韦弗 (Warren Weaver)，“科学与复杂性”，《美国科学家》 36 (1948)：539。

6. Warren Weaver, “Science and Complexity,” American Scientist 36 (1948): 539.

这个有组织的复杂性领域准确地描述了工程师在研究计算机网络时所面临的情况。无论从哪个角度处理计算机网络，问题都是复杂且有组织的。

This field of organized complexity exactly describes the situation engineers face in looking at computer networks. No matter what angle a computer network is approached from, the problem is both complex and organized.

• 设计协议时要牢记一组特定的目标、关于如何解决问题的特定思维方式，以及当前最佳使用、未来灵活性、可支持性和易于实施之间的一系列权衡。

• Protocols are designed with a specific set of goals in mind, a specific mindset about how the problems approached should be solved, and a set of tradeoffs between current optimal use, future flexibility, supportability, and ease of implementation.

• 在网络之上运行的应用程序在设计时考虑了一组特定的目标。

• Applications that run on top of a network are designed with a specific set of goals in mind.

• 提供使计算机网络正常工作的元数据的控制平面在设计时考虑了一组特定的目标。

• Control planes that provide the metadata that make a computer network work are designed with a specific set of goals in mind.

• 通过网络在各个级别传输信息的协议在设计时都考虑到了一组特定的目标。

• Protocols that carry information through the network, at every level, are designed with a specific set of goals in mind.

无论考虑计算机网络中的哪个系统（从协议到设计，从应用程序到元数据），每个系统都是根据一组特定的目标、关于如何解决当前问题的特定思维方式以及一组特定的权衡而设计的。其中一些可能是隐含的，而不是明确的，但它们仍然是有意的目标或目标。

No matter which system within computer network is considered—from protocols to design to applications to metadata—each one was designed with a specific set of goals, a specific mindset about how to solve the problems at hand, and a specific set of tradeoffs. Some of these might be implicit, rather than explicit, but they are, nonetheless, intentional goals or targets.

网络不仅仅是一个表现出有组织的复杂性的单个系统，而是许多不同的连锁系统，每个系统都表现出有组织的复杂性，并且所有这些系统组合起来也表现出一组目标（也许是一组更短暂的目标，例如”作为“使业务增长”，但仍然是一组目标）。

A network is not just a single system that exhibits organized complexity, but a lot of different interlocking systems, each of which exhibits organized complexity, and all of which combined exhibit a set of goals as well (perhaps a more ephemeral set of goals, such as “making the business grow,” but a set of goals nonetheless).

哲学旁白

A Philosophical Aside

在哲学领域，有人相信不存在有组织的复杂性这样的东西。相反，看似有组织的复杂性只是任何物理系统一旦变得足够复杂时就会出现的结果——这种组织以某种方式“内置”于自然秩序，或者内置于物质本身形成和相互作用的方式中。这一学派认为，任何和所有行为都可以追溯到某种物理原因（例如，人类实际上所做的决定并不像人类发生的决定那么多）。无论读者对这个主题的立场如何（在这里争论哲学问题超出了本书的范围），就网络架构而言，实际结果是：这并不重要。网络是由人们设计来解决一组特定问题的；无论这些设计“背后”是什么，为了理解计算机网络及其设计，我们都必须找到“原因”。为什么有人以这种特殊的方式设计这个？为什么有人要做出这种特殊的权衡？

Within the realm of philosophy, there are those who believe that there is no such thing as organized complexity. Instead, what appears to be organized complexity is simply the result of emergence within any physical system once it becomes complex enough—that organization is somehow “built in” to the natural order, or into the way matter itself is formed and interacts. This school of thought believes that any and all actions can be traced back to some physical cause (for instance, that humans do not actually make decisions as much as decisions happen to humans). Whatever the reader’s stand on this topic (and it is outside the scope of this book to argue the philosophical questions here), the practical result, in terms of network architecture, is: it doesn’t matter. Networks are designed by people to solve a particular set of problems; no matter what is “behind” these designs, we must, to understand computer networks and their designs, get to the “why.” Why did someone design this in this particular way? Why did someone make that particular tradeoff?

因此，网络复杂性不能按照传统意义上简单地测量、计算和“解决”。即使一切都可以在单个网络中测量，即使通过这种测量收集的所有信息都可以以某种有意义的方式进行处理，仍然不可能完全表达计算机网络的复杂性。本质上是因为无法衡量或表达意图。

Network complexity, then, cannot simply be measured, computed, and “solved,” in the traditional sense. Even everything could be measured in a single network, and even if all the information gathered through such measurement could be processed in a way that made some sense, it would still not be possible to fully express the complexity of a computer network in all its myriad parts—in essence because there is no way to measure or express intent.

这是浪费时间吗？

Is This a Waste of Time?

这并不意味着尝试测量网络复杂性是浪费时间。然而，它的真正含义是，必须以极大的谦逊态度来解决这个问题。工程师需要非常小心地理解设计每个部分中所做的权衡，并非常有意识地记住，准确预测任何特定设计决策的结果是有限的。

None of this means it is wasting time to attempt to measure network complexity. What it does mean, however, is that the problem must be approached with a large dose of humility. Engineers need to be very careful about understanding the tradeoffs being made in every part of the design, and very intentional in remembering that there are limits to accurately predicting the outcome of any particular design decision.

工程师不应“放弃”，而应尽一切可能来理解复杂性，控制复杂性，最小化复杂性，并做出明智的权衡——但现在没有、也永远不会有解决复杂性的灵丹妙药。正如第一章“定义复杂性”中所解释的，存在三组，其中只能选择两组，并且存在一些曲线，其中增加一个轴的复杂性以解决特定问题实际上会导致另一个轴出现问题。

Instead of “giving up,” engineers should do everything possible to understand the complexity, to contain it, to minimize it, and to make intelligent tradeoffs—but there isn’t, and won’t ever be a silver bullet for complexity. As explained in Chapter 1, “Defining Complexity,” there are sets of three out of which only two can be chosen, and there are curves where increasing complexity in one axis to solve a particular problem actually causes problems in another axis.

衡量和管理复杂性并不是浪费时间，除非您相信自己能够真正解决问题——因为“问题”无法“解决”。

Measuring and managing complexity is not wasting time unless you believe you can actually solve the problem—because the “problem” cannot be “solved.”

最后的想法

A Final Thought

迄今为止，这项复杂性调查得出的结论是：

This investigation of complexity has so far concluded:

• 解决难题需要复杂性，特别是在稳健设计领域。

• Complexity is necessary to solve difficult problems, particularly in the area of robust design.

• 超过一定水平的复杂性实际上会导致脆弱性——稳健但脆弱。

• Complexity beyond a certain level actually causes brittleness—robust yet fragile.

• 在系统层面以任何有意义的方式衡量复杂性是困难的（或者也许是不可能的）。

• Complexity is difficult (or perhaps impossible) to measure in any meaningful way at the systemic level.

• 有许多类别的问题无法解决三个目标中的两个以上（例如快速、廉价、高质量）。

• There are a number of classes of problems where it is impossible to resolve for more than two of three goals (such as fast, cheap, high quality).

考虑到这组要点，这似乎是路的尽头。网络工程师依赖于无法有效测量的东西，而测量始终是控制和管理问题集的第一步。最终，永远不会有一个单一的数字或公式可以描述网络的复杂性。我们是否应该简单地戴上海盗帽并宣称“放弃所有进入这里的人的希望”？或者有什么办法可以摆脱这个困境吗？

Given this set of points, it might seem like this is the end of the road. Network engineers are reliant on something that cannot be effectively measured—and measurement is always the first step in controlling and managing a problem set. There will never, in the end, be a single number or formula that can describe network complexity. Should we simply put on our pirate hats and proclaim, “Abandon hope all ye who enter here”? Or is there some way out of this corner?

事实上，有一种合理的方法来处理现实世界中的复杂性。与其试图找到绝对的“复杂性度量”，或者找到某种能够“解决”复杂性的算法，不如构建一种启发式方法，或者一种查看问题集的方法，从而找到解决方案。在这种情况下，启发式过程分为两部分。

There is, in fact, a reasonable way to approach complexity in the real world. Rather than trying to find an absolute “measure of complexity,” or find some algorithm that will “solve” complexity, it’s possible to construct a heuristic, or a method of looking at the problem set that will enable a path to a solution. The heuristic, in this case, is a two-part process.

首先，揭示网络设计中固有的复杂性权衡。揭示这些权衡将有助于工程师在为特定问题集选择任何特定解决方案时，就获得什么和失去什么做出明智的选择。每个问题不可能得到同样好的解决；在某一点应用的任何解决方案都会增加其他地方的复杂性。

First, expose the complexity tradeoffs inherent in network design. Exposing these tradeoffs will help engineers make intelligent choices about what is being gained, and what is being lost, when choosing any particular solution to a particular problem set. Every problem cannot be solved equally well; any solution applied at one point will increase complexity somewhere else.

换句话说，网络工程师需要学会有意识地对待复杂性。

To put it in other terms, network engineers need to learn to be intentional about complexity.

下一章将开始讨论三个特定的复杂领域——操作、设计和协议。在每种情况下，都会考虑设计师必须进行权衡的几个地方，以说明将复杂性公开化的过程。工程师在设计和管理网络时越有意识地做出有关复杂性的决策，就越有可能满足现实世界对网络的需求。

The next chapter will begin looking at three specific realms of complexity—operational, design, and protocol. In each of these cases, several places where designers must make tradeoffs to illustrate the process of bringing complexity out into the open will be considered. The closer engineers get to making intentional decisions about complexity when designing and managing networks, the more likely meeting the real-world demands placed on networks will be possible.

第 4 章操作复杂性

Chapter 4. Operational Complexity

本章分两个阶段讨论操作复杂性。第一部分探讨问题空间；第二部分考虑各种解决方案、它们如何解决复杂性问题以及每种解决方案所涉及的权衡。虽然这些部分并不详尽，但它们将概述如何寻找操作中的复杂性，以及如何思考各种可用解决方案的案例研究。

This chapter addresses operational complexity in two stages. The first section explores the problem space; the second considers the various solutions, how they address the complexity issues, and tradeoffs involved in each one. While these sections will not be exhaustive, they will provide an overview of where to look for complexity in operations, and case studies of how to think through the various solutions available.

探索问题空间

Exploring the Problem Space

本节考虑两个更大的主题，每个主题都有两个更具体的用例或研究领域。首先是人类与网络作为一个系统交互的成本。人与网络之间的交互超出了简单的用户界面部分，而是工程师可以通过一组心智模型、协议操作、业务和策略概念以及其他领域来理解网络的方式。特别是，策略在第二个主题中脱颖而出，这是网络设计中很少考虑的一个领域，即策略分散与网络中的最佳流量。

This section considers two larger topics, each with two more specific use cases or areas of investigation. The first is the cost of human interaction with the network as a system. The interaction between people and the network reaches beyond the simple user interface piece of the puzzle, and into the way in which engineers can understand the network through a set of mental models, protocol operations, business and policy concepts, and other areas. Policy, in particular, comes to the fore in the second topic, an area rarely considered in network design, policy dispersion versus optimal traffic flow through the network.

这里的示例并不是对该领域中各种问题的“结束一切”的描述，而是试图描述充分描述该领域的最小用例集。

The examples here are not an “end all, be all,” description of the various sets of problems in this space, but rather an attempt to describe a minimal set of use cases that describe the space adequately.

人类与系统交互的成本

The Cost of Human Interaction with the System

人类通过许多不同的工作流程与网络交互，包括设计、部署、管理和故障排除。虽然每个这些工作流程旨在实现一件事——在网络上部署新服务或应用程序——它们实际上都必须通过与分散在网络中的大量设备的交互来完成。可以推断出一些关键原则：

Humans interact with networks through a number of different workflows, including design, deployment, management, and troubleshooting. While each of these workflows is intended to result in one thing—the deployment of a new service or application on the network—they all must actually be completed through interaction with a large number of devices scattered throughout the network. A number of key principles can be inferred:

1.需要人类触摸才能执行结果的设备数量与该网络的操作复杂性相关。

1. The number of devices that need to be touched by a human to perform an outcome correlates to the operational complexity of that network.

2.这种运营复杂性直接转化为运营支出 (OPEX)。

2. This operational complexity directly translates into Operational Expenditures (OPEX).

3.降低运营复杂性将带来更精简、更高效、更高投资回报 (ROI) 的网络。

3. Reducing the operational complexity will result in leaner, more productive, higher Return on Investment (ROI) networks.

4.影响操作复杂性的设备数量包括直接“接触”的设备和“引用”的设备（例如在策略定义中）。

4. The number of devices affecting operational complexity includes both devices directly “touched” and devices “referenced” (as for example in a policy definition).

深入研究几个关键案例将有助于了解这种操作复杂性的根本原因，这种操作复杂性的症状是人机交互次数增加。

It will be useful to dive deeper into a couple of key cases to understand the root cause of this operational complexity that has the symptom of increased number of human interaction times.

在网络中应用策略

对于第一个用例，请考虑运营商在网络中的所有边缘设备上实施策略。理解这个问题的最简单方法是考虑运营商必须接触网络边缘的每个设备。这种手动部署新策略的方法（至少）存在四个主要问题：

For the first use case, consider an operator implementing a policy across all the edge devices in a network. The simplest way to understand this problem is to consider that the operator must touch each of the devices along the network edge. There are (at least) four major problems with such an approach to deploying this new policy manually:

• 要手动部署新策略，运营商需要接触网络中的每个边缘设备（可能有数千个）。这可能需要数千小时的网络工程工作，工程师可以花时间思考更富有成效的事情，例如下一波新设备或设计挑战。

• To deploy the new policy manually, the operator would need to touch each edge device in the network—potentially thousands of them. This could take thousands of hours of network engineering work, time that engineers could be thinking about more productive things, like the next wave of new equipment or design challenges.

• 随着部署此新策略所需的时间的推移，要求（以及策略）可能会发生变化。这并不总是一个明显的结果，但在现实生活中，新政策（或其他网络变化）的多次推出被中途停止以管理“即时暴政”，然后从未完成。结果是网络中的策略杂乱无章，以看似随机的方式部署在整个网络中。

• Over the time required to deploy this new policy, requirements (and hence the policy) could change. This isn’t always an obvious result, but there are real-life situations in which multiple rollouts of new policies (or other network changes) were stopped midstream to manage the “tyranny of the immediate,” and then never finished. The result is a network with a mishmash of policies deployed in a seemingly random fashion throughout the network.

• 即使新策略在一段时间内完全部署，部署期间网络也会处于不一致的状态。这可能会使网络故障故障排除变得困难，导致政策冲突造成积极损害（例如泄露有关客户的机密信息），并导致许多其他意想不到的副作用。

• Even if the new policy is fully deployed over some period of time, the network will be in an inconsistent state during the deployment. This can make it difficult to troubleshoot network failures, lead to conflicting policies causing positive harm (such as the release of confidential information about customers), and cause many other unintended side effects.

• 在长时间内配置大量设备时，人们很容易犯错误。网络设备配置中出现的错误导致网络中断的时间间隔称为平均错误间隔时间 (MTBM)。就像 MTBF 和 MTTR 一样，MTBM 可以被跟踪和管理，但大规模手动配置设备总会导致随着时间的推移，配置中出现错误。

• It’s common enough for humans to make a mistake when configuring a large number of devices over a long period of time. The amount of time between mistakes woven into the configuration of network devices causing a network outage can be called the Mean Time Between Mistakes (MTBM). The MTBM can, just like the MTBF and MTTR, be tracked and managed—but manually configuring devices on a large scale will always result in mistakes creeping into configurations over time.

大规模手动部署的复杂性

The Complexity of Large-Scale Manual Deployments

在大型网络中应用策略或其他新配置的规模可能令人望而生畏，例如，请考虑以下情况。多年来，一家大型企业一直尝试在本地园区网络上从一种路由协议 (IGRP) 转换为另一种路由协议 (EIGRP)。然而，这个校园网络极其密集，虽然它只包含大约一百个路由器，但它也恰好包含大约一千个互连这些路由器的低速链路。网络的某些部分是传统电路（例如T1），其他部分是帧中继，而其他部分是交换令牌环网段，还有一些是短期以太网网段。

The scale of applying a policy, or some other new configuration, within large-scale network can be daunting—as an example, consider the following situation. A large enterprise has, over a number of years, attempted to convert from one routing protocol (IGRP) to another (EIGRP) on a locally contained campus network. However, this campus network is extremely dense; while it contains only around a hundred routers, it also happens to contain about a thousand low-speed links interconnecting those routers. Some parts of the network are traditional circuits (such as T1s), others are Frame Relay, while others are switched Token Ring segments, and still others are short-run Ethernet segments.

由于 IGRP 和 EIGRP 不再像以前那样广为人知，因此需要注意一个奇点：当 EIGRP 和 IGRP 进程配置在具有相同自治系统编号的单个路由器上时，这两个进程会自动在它们之间重新分配，从而将两个协议之间的度量和其他信息，使得所有重新分配的路由看起来都是内部学习的。当网络工程组第一次尝试在两种协议之间切换时，他们通过简单地在同一 AS 中配置多个具有 EIGRP 和 IGRP 的路由器来使用此功能。

As IGRP and EIGRP aren’t as well known as they once were, it’s important to note a singular point: when EIGRP and IGRP processes are configured on a single router with the same autonomous system number, the two processes automatically redistribute between themselves, converting the metrics and other information between the two protocols such that all the routes redistributed appear to be internally learned. When the network engineering group first attempted to switch between the two protocols, they used this feature by simply configuring a number of routers with EIGRP and IGRP in the same AS.

在此过程中的某个时刻，网络崩溃了。恢复后，网络工程团队使网络处于半部署状态，部分网络在同一 AS 中同时运行 EIGRP 和 IGRP。此时，他们决定尝试不同的转换路径，配置 EIGRP在不同 AS 的不同路由器组上与 IGRP 并存并混合，并在它们之间手动重新分配。

At some point in this process, the network crashed. On recovering, the network engineering team left the network in the half-deployed state, with part of the network running both EIGRP and IGRP in the same AS. At this point, they decided to try a different path to conversion, configuring EIGRP alongside and intermingled with IGRP on a different set of routers, in different AS, and manually redistributing between them.

再说一次，在这个过程中的某个时刻过程中（第二次转换尝试），网络崩溃了。恢复后，网络工程团队再次使网络处于半部署状态。现在，所有路由器都在单个 AS 中运行 IGRP，一些路由器在 IGRP 之上配置了不同的 EIGRP AS，而其他一些路由器在同一 AS 中配置了 EIGRP。在转换网络的最后一次尝试中，网络工程人员从网络中的另一组路由器中删除了 IGRP，并在与网络中任何其他地方在短期计划中断期间部署的完全不同的 AS 中将其替换为 EIGRP。沿着网络的 EIGRP 和 IGRP 部分之间的边界，他们配置了两个协议之间的重新分配，以在转换过程中保持可达性。

Again, at some point in this process (the second conversion attempt), the network crashed. On recovering, the network engineering team again left the network in the half-deployed state. Now all the routers are running IGRP in a single AS, some of the routers are configured with a different EIGRP AS on top of IGRP, and some other set of routers are configured with EIGRP in the same AS. In a final attempt to convert the network, the network engineering staff removed IGRP from yet another set of routers in the network, replacing it with EIGRP in a completely different AS from that deployed anyplace else in the network during short interval planned outages. Along the border between the EIGRP and IGRP sections of the network, they configured redistribution between the two protocols to maintain reachability during the conversion process.

再次，网络失败；控制平面根本不会收敛。此时，情况就很糟糕了。网络中有四个部分，一个部分运行 IGRP，另一个部分在同一 AS 中与 IGRP 一起运行 EIGRP（通过相同的链路路由到相同的目的地），另一个部分在不同的 AS 中与 IGRP 一起运行 EIGRP，另一个部分在完全独立的 AS 中运行 EIGRP。不同AS相互重新分配。更令人困惑的是（好像这还不够），最初的网络规划是由多个 IGRP AS 规划的，每个 IGRP AS 相互重新分配，而不是单个 IGRP AS。在某个时刻，网络最终导致每个路由器都具有接近唯一的配置——因为在整个网络中部署新的、一致的路由协议的三次尝试都失败了。

Again, the network failed; the control plane simply would not converge. At this point things were dire. There were four sections in the network, one running IGRP, another running EIGRP alongside IGRP (routing across the same links for the same destinations) in the same AS, another running EIGRP alongside IGRP in a different AS, and another running EIGRP in a completely different AS with mutual redistribution. To add to the confusion (as if this weren’t enough), the original network plan was laid out with multiple IGRP AS, each mutually redistributed into one another, rather than a single IGRP AS. At some point, the network ended up with each router having a close to unique configuration—because of three failed attempts to deploy a new, consistent routing protocol throughout.

解决方案？一组工程师通过网络逐跳远程登录，删除了网络中每台路由器上的所有路由进程。一旦所有动态路由被删除，网络就会被重建，再次使用 Telnet 逐跳，在整个网络的单个通用 AS 中配置 EIGRP。

The solution? Telnetting hop-by-hop through the network, a group of engineers removed every routing process on every router in the network. Once all dynamic routing had been removed, the network was rebuilt, again using Telnet hop-by-hop, configuring EIGRP in a single, common, AS throughout the entire network.

教训是：大规模手动部署很困难。当部署遇到困难并且网络出现故障时，立即的暴政就会突然出现。大门必须打开，业务必须带来收入，因此网络必须足够稳定才能使关键任务应用程序再次运行，并且尝试被遗弃了几天（很容易变成几个月甚至几年）。最终结果往往是一团混乱的失败部署、难以管理且无法排除故障。有时，唯一的解决办法就是“重新开始”。

The lesson is this: manual deployments at scale are hard. When the deployment hits the wall and the network fails, suddenly the tyranny of the immediate kicks in. The doors must open, and the business must bring in revenue, so the network is stabilized enough to get mission critical applications running again, and the attempt is abandoned for a few more days (which can easily turn into months and years). The end result is often a mess of failed deployments, difficult to manage and impossible to troubleshoot. Sometimes the only solution is simply to “start over.”

排除网络故障

第二个感兴趣的用例是对网络故障进行故障排除，这直接影响网络可用性的衡量标准之一，即 MTTR。对大型系统进行故障排除的过程通常既是一门艺术，也是一门工程技能，包括多个阶段或工作领域：

A second use case of interest is troubleshooting a network failure, which directly impacts one of the measures of network availability, the MTTR. The process of troubleshooting a large-scale system is often as much an art as it is an engineering skill, including multiple phases or areas of work:

• 问题识别，通常涉及某种形式的半分裂以及预期（或理想）状态与实际状态的比较。识别问题通常会花费一半以上的故障排除过程（重要的是不知道用锤子敲什么，重要的是知道在哪里敲）。

• Problem identification, which normally involves some form of half splitting and comparison of expected (or ideal) state with actual state. Identifying the problem often consumes more than half the troubleshooting process (it’s not knowing what to strike with the hammer, it’s knowing where to strike it that matters).

• 问题修复，通常涉及更换或重新配置设备。通常，这是以提供临时而非永久修复的方式进行的。请注意，虽然临时修复已到位，但网络仍会遇到上述策略部署场景中的相同问题 - 如果即时的暴政占据上风，并且临时修复从未被替换或验证，则网络可能会建立起来稍后会导致大量问题的“修复层”或技术债务。

• Problem remediation, which normally involves replacing or reconfiguring the device. Often this is undertaken in a way that provides a temporary, rather than permanent, fix. Note that while a temporary fix is in place, the network is subject to the same problems described above in the policy deployment scenario—if the tyranny of the immediate takes over, and the temporary fix is never replaced or verified, the network can build up a “layer of fixes” or technical debt that causes a large number of problems later on.

• 根本原因分析，通常涉及更深入地研究问题症状、临时解决方案以及从网络收集的任何进一步信息，以比“这就是问题”更深入。根本原因分析会查找何时进行导致问题的更改，以及为什么任何更改管理流程都没有捕获错误。根本原因分析的目的是验证临时修复是否是正确的解决方案或将其替换为更永久的解决方案，并尝试确保问题将来不会再次发生。

• Root cause analysis, which normally involves taking a deeper look at the problem symptoms, the temporary fix, and any further information gathered off the network to go deeper than “this is the problem.” Root cause analysis looks for when changes that caused problems were made, and why any change management process didn’t catch the error. The point of root cause analysis is to verify that the temp fix is the correct solution or replace it with a more permanent solution, and to try and ensure the problem doesn’t occur again in the future.

打击地点：工程师和锤子的虚构故事

Where to Strike: The Apocryphal Story of the Engineer and the Hammer

对于不熟悉上面第一个要点中工程师和锤子的虚构故事的读者，这里重复了一种形式。曾经有一位工程师在一台特定的机器上工作了二十年，然后决定退休。在没有他的情况下，这台机器工作了一段时间，但有一天出现了某种无人能诊断或修复的问题。引进了很多顶尖专家，最后公司放弃了，重新聘请了这位退休人员作为顾问。到达现场后，他听了一会儿机器发出的噪音，拿起一把锤子，然后打过一次。噪音消失了，机器开始正常运转。几天后，该公司收到了一张 10,000 美元的账单。会计部门表示反对，并向他发送了一封电子邮件，表示他们需要了解如何拿起锤子敲击机器才真正值这么多钱。退休人员发回一份新账单，这次是逐项列出的。上面写着：“1 美元：用锤子敲击机器。9,999 美元：知道用锤子敲打机器的位置。”

For readers unfamiliar with the apocryphal story of the engineer and the hammer from the first bullet point above, one form is repeated here. There was once an engineer who worked on a particular piece of machinery for twenty years, and then decided to retire. The machine worked without him fine for some time, but then one day developed some sort of problem no one could diagnose or repair. Many top experts were brought in, until finally the company gave up and hired the retiree back in as a consultant. On arriving at the site, he listened for a moment to the noise the machine was making, picked up a hammer, and whacked it once. The noise went away, and the machine began operating properly. A few days later the company received a bill for $10,000. The accounting department objects, sending him an email stating they need to understand how picking up a hammer to hit the machine is really worth this amount of money. The retiree sends back a new bill, this time itemized. It says, “$1: hitting machine with hammer. $9,999: knowing where to hit machine with hammer.”

在解决复杂性网络问题时，有几个要点会发挥作用，包括：

Several points come into play when troubleshooting a network problem in regard to complexity, including:

• 操作员必须接触才能解决问题的设备数量。这在范围和概念上与操作员部署策略必须接触的设备数量相似。

• The number of devices the operator must touch to troubleshoot the problem. This is similar in scope and concept as the number of devices the operator must touch to deploy a policy.

• 网络中可进行测量的位置数量以及进行这些测量的难度。

• The number of places measurements can be taken in the network, and how difficult it is to take those measurements.

• 可用于定义网络正常情况的信息量，从而回答“发生了什么变化？”的问题。

• The amount of information available to define what the network normally looks like, and hence answer the question, “what’s changed?”

为了正确理解这些，请考虑一个特定的用例。考虑一个扁平 IP 网络，其中源节点失去与目标节点的连接。在此示例中，假设源和目标之间有一些设备。负责解决此问题的工程师可能会访问每个设备（例如，从默认网关开始，沿着流量应经过的路径通过路由器逐跳访问每个设备）并发出许多命令来了解节点的状态并确定路径中的哪个节点导致中断。随着网络规模和复杂性的增长，通过网络的路径变得更难以追踪，并且实时跳过的设备数量可能变得几乎不可能（当错误发生时）。如果网络超出单个管理域，则由于访问限制，从路径上的每个设备收集信息可能变得不可能。网络规模越大、越复杂，使用手动方法进行故障排除就越困难。

To put these into perspective, consider a specific use case. Consider a flat IP network in which a source node loses connectivity to a destination node. In this example, assume there are a few devices between source and destination. The engineer tasked with troubleshooting this problem could potentially access each device (say by accessing each device hop-by-hop through the routers along the path through which the traffic should travel, starting with the default gateway) and issue a number of commands to understand the node’s state and determine which of the nodes in the path is causing the outage. As the network grows in scale and complexity, the path through the network becomes harder to trace, and the number of devices to hop through may become close to impossible in real time (as the error is occurring). If the network grows beyond a single administration domain, gathering information from each device along the path may become impossible due to access restrictions. The larger and more complex the network is, the more difficult it will be to troubleshoot using manual methods.

策略分散与最佳流量处理

Policy Dispersion Versus Optimal Traffic Handling

网络设计者通常不会考虑控制平面复杂性和最佳利用率之间的关系，但通过策略分散的概念存在明显的联系。政策分散只不过是其中之一如上一节所述，人类与网络交互所涉及的成本 - 需要配置以实施特定策略的设备数量。一个具体的例子将有助于理解问题的范围。本例使用图 4.1中的网络。

Network designers don’t often think about the relationship between control plane complexity and optimal utilization, but there is a clear link through the concept of policy dispersion. Policy dispersion is nothing more than one of the costs involved in the human interaction with the network, as outlined in the previous section—the number of devices that require configuration to implement a specific policy. A specific example will help in understanding the scope of the problem. Use the network in Figure 4.1 for this example.

图 4.1 策略分散和最佳网络利用率

Figure 4.1 Policy Dispersion and Optimal Network Utilization

假设需要对传入流量应用三组策略：

Assume three sets of policies need to be applied to incoming traffic:

• 服务质量策略对源自主机A 的大文件传输进行分类，以便将这些流放入沿路径的较低分类队列中。

• A quality of service policy classifying large file transfers originating from Host A so these flows are placed into a lower classification queue along the path.

• 服务质量策略对源自主机B 的较小文件传输进行分类，以便将这些流放置在沿路径的中等优先级队列中。

• A quality of service policy classifying smaller file transfers originating from Host B so these flows are placed in a medium priority queue along the path.

• 服务质量策略将来自连接到“边缘”框中任何路由器的任何主机的语音流量分类到高优先级队列中。

• A quality of service policy classifying voice traffic originating from any host connected to any router in the “Edge” box into a high priority queue.

这里关注的不是如何实施这些政策，而是在哪里实施这些政策，以及在决定在哪里实施这些政策时所涉及的权衡。这些策略可以在所示网络中的三个位置实施：

The concern here is not with how these policies might be implemented, but where they might be implemented—and the tradeoffs involved in deciding where to implement them. There are three places these policies can be implemented in the network illustrated:

• 部署这些策略的最自然位置是网络边缘（示例中的路由器 C 和 D），并且可能是层次结构中同一级别的所有其他设备。此设计的优点是在边缘设备与路由器 E、F、G 和 H（以及网络的其余部分）之间的链路上强制执行策略，因此整个路径上的流量得到最佳处理。此解决方案的缺点是，边缘路由器的配置变得更加复杂，因为添加了额外的配置来实施策略，并且（可能）存在大量边缘设备（在本例中只有十个，但数量很大）规模网络可能有数千个）。

• The most natural place to deploy these policies would be along the edge of the network, (Routers C and D in the example), and potentially all the other devices at the same level in the hierarchy. This design has the advantage of enforcing the policy on the links between the edge devices and Routers E, F, G, and H (and the rest of the network), so traffic is handled optimally all along the path. The disadvantage of this solution is the configuration of the edge routers becomes more complex as additional configuration is added to implement the policy—and there are (potentially) a large number of edge devices (in this case there are only ten, but in large-scale networks there could be many thousands). The number of devices on which the policy must be configured can be traded off against the uniformity of the configuration for all edge devices throughout the network, of course, but this doesn’t “solve” the complexity problem, it just moves the complexity from one place to another in the operational chain.

• 第一个替代方案是在路由器 E、F、G 和 H 上实施这些策略。在这种情况下，必须同步策略的路由器数量要少得多，但边缘路由器与这四个路由器之间的链路主机生成的流量不会以最佳方式使用路由器。代价是只有四个路由器必须在其上实施和维护策略，从而使策略的实施和同步更加容易。

• The first alternative would be to implement these policies at Routers E, F, G, and H. In this case, the number of routers across which the policies must be synchronized is much smaller, but the links between the edge routers and these four routers will not be used optimally by the traffic flows being generated by the hosts. The tradeoff is there are only four routers on which the policies must be implemented and maintained—making the implementation and synchronization of policies easier.

• 第二种选择是在路由器M 和N 上实施这些策略，因此它们主要控制沿着将目标主机连接到网络的以太网链路流动的流量。这会增加每条未达到最佳使用效果的路径的长度，并且还会减少必须配置和管理策略的设备数量。

• The second alternative would be to implement these policies at Routers M and N, so they primarily control the traffic flowing along the Ethernet link connecting the destination host to the network. This increases the length of each path that will be used suboptimally, and also decreases the number of devices on which the policies must be configured and managed.

在这种情况下，答案可能看起来很简单——沿网络边缘配置策略。然而，在具有多个边缘模块的大型网络中，选择变得更加复杂。

The answer, in this case, might appear to be simple—configure the policies along the edge of the network. In larger networks with a number of edge modules, however, the choice becomes more complicated.

如果有数千台边缘设备，并且每台边缘设备都配置相同的策略，以保证应用和配置的一致性，以及作为整个网络的最佳网络使用，有多少设备必须具有同步配置，以及错误的成本是多少？如果每个边缘设备仅配置其所需的策略，那么维护一千种不同配置的成本是多少，以及错误的成本是多少？

If there are thousands of edge devices, and each edge device is configured with the same policies to ensure consistency of application and configuration, as well as optimal network usage throughout the network, how many devices must have synchronized configurations, and what is the cost of a mistake? If every edge device is configured with just the policy it needs, then what is the cost of maintaining a thousand different configurations, and what is the cost of a mistake?

网络的哪些特定区域需要实施最佳流量，为什么？在这个小（且人为的）示例中，边缘路由器与路由器 E、F、G 和 H 之间的链路真的需要进行策略控制吗？简单地在层次结构的第一层上投入带宽来解决服务质量问题，以便可以在网络中的更少数量的设备上实施相关策略，是否会更有效？

Which specific areas of the network need to enforce optimal traffic flow, and why? In this small (and contrived) example, do the links between the edge routers and Routers E, F, G, and H really need to be policy controlled? Would it be more efficient to simply throw bandwidth at the problem of quality of service along the first layer of hierarchy so the relevant policies can be implemented on a smaller number of devices further into the network?

这些问题并不像乍看起来那么容易回答。要了解原因，请转向网络设计中使用的一些机制来管理之前考虑的操作复杂性。

These questions aren’t as easy to answer as they might first appear. To see why, turn to some of the mechanisms used in network design to manage the operational complexities previously considered.

解决管理复杂性问题

Solving the Management Complexity Problem

如果您是一名经验丰富的网络工程师，那么当您到达这一点时，您脑海中应该有一个声音尖叫“自动化！” 事实上，人类与网络交互以及通过网络进行策略分散的问题可以通过自动化管理工具来解决——在许多情况下这无疑是一个不错的选择。然而，网络设计和工程中存在一种将复杂性越过隔间墙转移到网络操作上的历史趋势，这意味着“网络管理系统将解决该问题”。将复杂性从网络设计或协议转移到网络管理系统并不能真正降低复杂性；相反。它只是将其隐藏在部门部门（或隔间）墙后面。

If you’re an experienced network engineer, by the time you get to this point, a voice in your head should be screaming “automate it!” The problems of human interaction with the network and policy dispersion through the network are, in fact, addressable through automated management tools—and this is certainly a good option in many cases. However, there is a historical tendency in network design and engineering to throw complexity over the cubicle wall onto the network operations, implying “the network management system will manage that problem.” Moving complexity from the network design or the protocol into the network management system doesn’t really reduce complexity; it just hides it behind a department division (or cubicle) wall.

当网络工程师和设计人员在尝试降低复杂性时仅考虑整个网络的一部分，而不是采用系统架构方法（其中复杂性在模块之间的转移非常明显）时，这种趋势会进一步加剧。只有通过查看整个系统，您才能看到消除某个离散区域的复杂性所产生的全部影响。

This tendency is further exacerbated when network engineers and designers consider only a part of the overall network when trying to reduce complexity, instead of taking a systemic architectural approach in which the moving of complexity from module to module is quite evident. Only by looking at the overall system can you see the full impact of removing complexity from one discrete area.

管理操作复杂性的三种不同机制可以更详细地说明这一点。自动化，因为它是所有大型网络最喜欢的工具，所以会首先考虑，其次是模块化，最后增加协议复杂性以降低管理复杂性。

Three different mechanisms for managing operational complexity can illustrate this in more detail. Automation, because it is the favorite tool of all large-scale networks, will be considered first, followed by modularity, and finally adding protocol complexity to reduce management complexity.

自动化作为管理复杂性的解决方案

Automation as a Solution to Management Complexity

考虑上一节中给出的示例——部署一组服务质量策略。在这种情况下的基本选择是：

Consider the example given in the previous section—deploying a set of quality of service policies. The basic choice in that situation was to either:

• 在更多设备上更靠近网络边缘部署策略，从而更优化地利用可用带宽，同时在更多设备上管理策略。

• Deploy the policy closer to the network edge on more devices, thus making more optimal use of the available bandwidth, but managing the policy on a larger number of devices.

• 将策略部署在更靠近网络核心的位置，减少必须管理策略的设备数量，但（可能）会降低可用带宽的使用效率。

• Deploy the policy closer to the core of the network, reducing the number of devices on which the policy must be managed, but (potentially) using the available bandwidth less efficiently.

因此，可用的解决方案指出了管理复杂性和最佳网络利用率之间的权衡（这种权衡实际上在网络设计决策中很常见）。自动化似乎提供了摆脱这种情况的“第三种方式”，允许将策略放置在边缘设备上而无需人工干预，从而解决复杂性权衡问题；然而，事情并没有那么简单。复杂性曲线没有“灵丹妙药”。以下各节将考虑权衡。

Hence, the solutions available point to a tradeoff between managing complexity and optimal network utilization (a tradeoff that’s actually quite common in network design decisions). Automation appears to provide a “third way” out of this situation, allowing the policy to be placed on the edge devices without manual intervention, thus resolving the complexity tradeoff; however, it’s not that simple. There is no “silver bullet” for the complexity curve. Tradeoffs will be considered in the following sections.

脆性

自动化系统使“人类不再做出反应”。虽然这确实提供了管理流程的一致性，但它也使管理流程中无法识别问题和议题。这种人类互动的消除可能会导致所谓的“骨化”（该词用于描述物质变成化石的过程）。结果可能是一种坚硬但易碎的材料——坚固但脆弱是第 1 章“定义复杂性”中用来描述这一点的术语。

Automated systems take the “human out of the reaction.” While this does provide consistency in the management process, it also takes the recognition of problems and issues out of the management process, as well. This removal of human interaction can result in what might be called ossification (the word used to describe the process of material becoming fossilized). The result can be a hard, but brittle, material—robust yet fragile is the term used to describe this in Chapter 1, “Defining Complexity.”

自动化是脆弱的，因为自动化流程无法响应每种可能的故障模式或情况，因为此类系统的人类设计者无法想象每种可能的故障模式或情况。自动化系统的脆弱性可以通过定期的人工监督和更好的设计来抵消，但必须对这种交互进行管理，从而增加部署旨在降低复杂性的系统过程中的复杂性。在考虑自动化解决方案来解决管理复杂性问题时，需要进行明显的权衡。

Automation is brittle because automated processes can’t respond to every possible failure mode or situation—because the human designers of such systems can’t imagine every possible failure mode or situation. The brittleness of an automated system can be offset by periodic human oversight and better design, but such interactions must be managed, increasing complexity in the process of deploying a system designed to reduce complexity. There is a clear tradeoff here that needs to be managed when considering automated solutions to solve management complexity problems.

当部署编排系统来管理数据中心结构中的隧道覆盖时，可能会看到这种情况的一个示例。应用程序可以相当容易地通过编排系统在结构上建立新的隧道路径——随着时间的推移，可以在没有任何人工干预的情况下构建数千（或数万）或这样的隧道。但是，当应用程序构建足够的隧道来克服任何特定设备的转发表管理覆盖在拓扑上的隧道集的能力时，会发生什么？最好的情况是隧道创建过程失败，导致操作员查看情况并尝试找出发生了什么。最糟糕的是由于转发位置超载而导致网络完全故障。

An example of this situation might be seen when deploying an orchestration system to manage tunneled overlays in a data center fabric. Applications can be given the ability to bring up new tunneled paths across the fabric through the orchestration system fairly easily—over time thousands (or tens of thousands) or such tunnels can be built without any human intervention. But what happens when an application builds enough tunnels to overcome the ability of any particular device’s forwarding table to manage the set of tunnels overlaid onto the topology? The best case is a tunnel creation process fails, causing a human operator to look at the situation and try to find out what’s going on. The worst is a complete network failure due to an overloaded forwarding place.

僵化还会以另一种方式导致脆性——假设“如果它存在，就有人需要它”。在纯自动化解决方案中，通常很难通过各种系统日志文件追踪特定隧道（在本示例中）的配置原因。运营商如何确定不存在因不正确的清理（或垃圾收集）过程而导致不必要的网络资源浪费的未使用隧道组不断增加？

Ossification also leads to brittleness in another way—the assumption that “if it exists, it is needed by someone.” In a purely automated solution, it is often difficult to trace down why a particular tunnel (in this example) was configured through the various system log files. How can the operator be certain there isn’t an ever increasing set of unused tunnels that are needlessly wasting network resources due to incorrect cleanup (or garbage collection) processes?

网络状态

从某种意义上说，自动化系统是网络状态的抽象。网络运营商不是直接处理任何给定设备的配置或当前运行状态，而是处理一个将预期状态解释为实际状态的软件。这种附加的抽象层使网络自动化能够消除大量设备上的复杂配置。

Automation systems are, in a sense, abstractions of the network state. Rather than dealing directly with configurations or the current running state of any given device, the network operator deals with a piece of software that interprets intended state into actual state. This added layer of abstraction is what gives network automation its power to cut through complex configurations on a large number of devices.

然而，这种抽象的另一面正在与网络的真实状态失去联系。查看单个设备的配置的工程师可能需要跟踪许多流程和代码片段，以了解特定配置的方式和原因。在大规模中断中，或者当追踪网络中断的时间非常宝贵时，这可能会出现问题。同样，这个问题可以通过仔细处理文档来解决；然而，记录在特定设备上安装特定配置的过程本身就是一种复杂性。

The other side of this abstraction, however, is losing touch with the real state of the network. An engineer looking at the configuration of a single device might need to trace through a number of processes and pieces of code to understand how and why a particular configuration is the way it is. This can be problematic in large-scale outages, or when time is at a premium in tracing down a network outage. Again, this can be resolved by taking great care with documentation; however, documenting the processes whereby a particular configuration is installed on a particular device is a form of complexity on its own.

该领域的第二个问题是无法单一来源，或者无法获得关于网络配置应该是什么以及为什么以这种方式配置的规范事实来源（将意图与部署联系起来）。工程师必须首先检查网络中部署的设备的实际配置，以了解实际配置和预期配置。他们可能会先查看自动化框架中的注释，然后最后查看存储在存储库中的某种形式的实际文档。随着时间的推移，存储在不同位置并在不同时间范围内进行管理的文档存在明显的差异趋势。由于没有“单一事实来源”，因此很难准确地知道任何特定配置的设计目的是什么，也很难知道它实际上应该如何部署。因为这，

A second problem in this space is the failure to single source, or to have a canonical source of truth about what the network configuration should be and why it’s configured this way (connecting intent to deployment). Engineers are bound to examine the actual configuration of the devices deployed in the network first to understand what is actual and intended. They might look at the comments in an automation framework second, and then finally into some form of actual documentation stored in a repository. There is a clear tendency for documentation stored in different locations, and managed within different time scales, to diverge over time. With no “single source of truth,” it’s hard to know precisely what any particular configuration was designed to do, nor how it should actually be deployed. Because of this, well-documented systems can actually provide a false view of the state of the network—a situation sometimes worse than having no (or minimal) documentation.

管理管理系统

任何从软件角度从事过大规模系统开发的人都可以告诉您所涉及的困难 - 跟踪代码库的更改、在代码投入生产之前测试代码、管理整个功能开发过程 - 每一个都有其自己的困难一系列需要解决的复杂问题。当一个网络自动化项目最初是几个网络工程师在白板上的一个“副项目”，很难想象几年后的最终系统，包括版本控制、全面的冒烟测试系统、实验室环境等等。

Anyone who has ever worked on a large-scale system development from a software perspective can tell you the difficulties involved—tracking changes to the codebase, testing the code before it reaches production, managing the entire feature development process—each of these has its own set of complex problems to solve. When a network automation project begins as a “side project” on the whiteboard with a couple of network engineers, it’s hard to imagine the final system, years later, with version control, a full blown smoke test system, a lab environment, and so on.

虽然自动化对于控制网络设备的复杂性来说是一个巨大的好处，但它本身也可能成为无限复杂性的根源。多年来，供应商一直承诺让自动化系统更轻松地管理设备。开放接口和收入预测之间不断摇摆。也许软件定义网络和其他更新的模型将提供解锁更简单的自动化管理的钥匙，但网络工程师应该意识到在这一领域存在明确的权衡。

While automation can be a huge benefit for controlling the complexity of network devices, it can also be a source of endless complexity on its own. Vendors have, for years, promised to make the process of managing equipment easier for automation systems; there is a constant swing between open interfaces and revenue projection. Perhaps software defined networks and other, newer, models will provide the keys to unlocking simpler automated management, but there are definite tradeoffs in this area the network engineer should be aware of.

关于自动化作为解决方案的最终想法

网络设备配置的自动化可能被视为类似于网络中许多其他任务的自动化。每个解决方案都有优点和缺点，并且每个解决方案都有其自己的复杂性。

Automation of network device configuration might be seen as similar to automating many other tasks in a network. Each solution has plusses and minuses, and each solution has its own set of complexities.

前面讨论的三个方面——脆弱性、网络状态和管理管理系统——是决定自动化任何特定网络配置任务以及构建自动化系统时需要考虑的因素。

The three areas discussed previously—brittleness, network state, and managing the management system—are factors that need to be considered when deciding to automate any particular network configuration task, and when building automation systems.

模块化作为管理复杂性的解决方案

Modularity as a Solution to Management Complexity

模块化通常用作将网络划分为多个故障域的机制，但它也是管理操作复杂性的有用机制。模块化网络有很多好处，包括能够限制网络结构的范围，限制网络变化的（不利）影响，并允许模块不仅可重用，而且可以在一定程度上独立发展。因此，模块化网络减少了元素、协议的数量以及人类在与这些网络交互时必须处理的信息量。

Modularity is often used as a mechanism to break networks up a number of failure domains, but it’s also a useful mechanism to manage operation complexity. Modularizing networks has many benefits, including the ability to constrain the scope of network structures, constrain the (adverse) effects of network changes, and allow for modules that are not only reusable but also can evolve somewhat independently. As a corollary, modularizing networks reduces the number of elements, protocols, and the amount of information that humans have to deal with when interacting with these networks.

返回到前面的示例，确定是将一组服务质量策略分发到广泛的设备上的网络边缘，还是更靠近网络核心的较小的一组设备上。该示例中考虑的问题之一是以下任一问题：

Returning to the previous example, determining whether to distribute a set of quality of service policies to the edge of a network on a wide array of devices, or on a smaller set of devices closer to the network core. One of the issues considered in that example was the problem of either:

将策略分发到每个边缘设备，从而将策略配置分发到大量不需要策略的设备

Distributing the policy to every edge device, and thus distributing the policy configuration to a large number of devices where the policy isn’t needed

或者

仅将策略分发到需要策略的一组设备，然后处理整个网络中缺乏一致性的问题

Distributing the policy only to the set of devices where the policy is needed, but then dealing with a lack of consistency throughout the network

模块化通过将设备分解为模块来帮助解决这个问题。然后，每个模块可以具有一组统一应用于该模块中所有边缘设备的策略。只要检查设备的网络工程师能够找到配置该设备的模块，一组通用或标准的配置就应该是显而易见的。

Modularity helps to resolve this problem by breaking devices up into modules. Each module can then have a set of policies that are applied uniformly across all the edge devices in the module. So long as a network engineer examining the device can locate the module in which the device is configured, a common or standard set of configurations should be readily apparent.

事实上，从这个意义上说，模块化是网络配置自动化所必需的服务类型抽象的附属物。将每个设备视为网络自动化系统中的“单一案例”，会将数千种配置的复杂性重新导入到自动化系统本身中，只有当可以分离出相对少量的“设备类别”时，网络自动化才有意义。其中可以抽象为单个配置。这就是模块化有助于实现的目标——设备不仅可以被分类为边缘或核心设备，还可以被分类为连接用户或连接到会计系统的进程的边缘设备（因为该设备位于主要由会计办公室使用的边缘模块）。

In fact, in this sense modularity is an adjunct to the abstraction of service types necessary to the automation of network configuration. Treating each device as a “single case” in a network automation system imports the complexity of thousands of configurations back into the automation system itself—network automation only makes sense if it’s possible to separate off a relatively small number of “devices classes,” each of which can be abstracted into a single configuration. This is what modularity helps to achieve—not only can a device be classified as an edge or core device, it can be classified as an edge device connecting a user or process that connects to accounting systems, for instance (because the device is located in an edge module used primarily by the accounting office).

这种更高的精度允许在整个网络中设计和使用更多而不是更少的“千篇一律”配置。从操作复杂性的角度来看，这个想法是能够将这些设备角色分组到模块和模板中，以便进一步能够与聚合进行交互。虽然运营商通常不会说“将策略X应用到所有边缘设备”，但他或她可能会说“将策略X应用到所有提供第 3 层虚拟专用网络 (L3VPN) 服务且数量超过Y 的边缘设备”订户数量。

This greater degree of precision allows more, rather than less, “cookie cutter” configurations to be designed and used throughout the network. The idea, from an operational complexity perspective, is to be able to group in modules and templates those device roles, to further be able to interact with the aggregate. While an operator typically will not be saying “apply policy X to all edge devices,” he or she could be saying “apply policy X to all edge devices” that provide Layer 3 Virtual Private Network (L3VPN) service and have more than Y number of subscribers.

然而，模块化并非没有复杂性的权衡。模块化可以在以下三个方向之一达到极端：

Modularity, however, is not without its complexity tradeoffs. Modularity can be taken to extremes in one of three directions:

•首先，模块化可能会被发挥到极致，试图为每种可能的极端情况构建完美的配置。如果采用这条路径，最终的模块大小通常为 1。但结果不是将每个设备视为单独的配置，而是在策略之上分层策略，在异常之上叠加异常，以至于最终的设备配置不仅是唯一的，而且是通过唯一的方法唯一创建的。一套规则。每个规则可能适用于多种设备，但每个设备都代表这些规则的唯一组合。这实际上导致的问题比它解决的问题还要多，因为它将交互界面的另一个完整领域引入到网络中——策略到策略，或域到域。

• First, modularity might be taken to the extreme of attempting to build the perfect configuration for every possible corner case. If this path is taken, the final module size is often one. But rather than treating each device as an individual configuration, the result is a layering of policy on top of policy, and exception on top of exception, to the point where the final device configuration is not only unique, it is uniquely created through a unique set of rules. Each rule might apply to a wide array of devices, but each device represents a unique combination of those rules. This actually causes more problems than it solves, as it introduces another entire realm of interaction surfaces into the network—policy to policy, or domain to domain.

其次，模块化可能会被发挥到极致，以牺牲使用网络的实际应用程序和服务为代价，将一系列可能的策略和配置管理到最低限度。在这种情况下，应用程序开发人员或用户可能会被告知他们无法在网络上部署特定服务或应用程序，因为这太麻烦了。很难围绕它调整网络策略。另一种说法是，“该应用程序可以部署，但该应用程序将以次优方式运行，因为没有简单的方法来调整我们的工具和模块以支持优化的策略要求。” 当然，总会有一些应用程序在具有一小组一致策略的网络上运行不佳，但也必须始终在策略的复杂性和所使用的抽象的“泄漏性”之间寻求平衡模块化网络以及网络上运行的实际应用程序和服务。

• Second, modularity might be taken to the extreme of managing down the array of possible policies and configurations to the minimum at the expense of the actual applications and services using the network. In this case, an application developer or user might be told they cannot deploy a particular service or application on the network because it would be too difficult to adjust the network policy around it. Another way of putting this would be, “this application can be deployed, but the application will run in a suboptimal way because there’s no easy way to adjust our tools and modules to support the policy requirements for optimization.” There will, of course, always be some set of applications that will run suboptimally on a network with a small set of consistent policies, but there must also always be a balance sought between the complexity of policy, the “leakiness” of the abstractions used to modularize the network, and the actual applications and services running on the network.

•最后，模块化可以发挥到完美干净的 API 或完美交互界面的极致，从而创建无法打破的硬面孤岛。这会通过网络设计中的僵化产生另一种形式的脆性；一旦竖井建立起来，网络就没有空间或方式来发展和解决新的业务问题。事实上，企业成为网络运营领域复杂性降低的受害者——这可能是“将复杂性扔到立方体墙上”的一个主要反面例子。

• Finally, modularity can be taken to the extreme of perfectly clean APIs, or perfect interaction surfaces, creating hard sided silos that cannot be broken down. This produces another form of brittleness through ossification in the network design; once the silos are set up, there is no space or way for the network to grow and resolve new business problems. The business becomes, in effect, a victim of complexity reduction in the network operations realm—perhaps a prime negative example of “throwing complexity over the cube wall.”

协议复杂性与管理复杂性

Protocol Complexity versus Management Complexity

现在回到排除网络、应用程序或服务故障时的人机交互问题。假设网络运营商需要确定两个特定主机之间的流量采用什么路径。操作员确定这一点的一种方法是从主机开始，确定该特定主机的默认网关。一旦到达默认网关，操作员就可以检查本地转发表以查找下一跳；移动到下一跳设备，操作员可以跟踪通过网络的流量路径。

Return, for the moment, to the problem of human interaction in troubleshooting a network, application, or service failure. Assume the network operator needs to determine what path is traffic taking between two specific hosts. One way the operator could determine this is to start at the host, determining the default gateway for that specific host. Once at the default gateway, the operator could then examine the local forwarding table to find the next hop; moving to this next hop device, the operator could trace the path of the traffic through the network.

或者，操作员可以仅使用来自源节点的跟踪路由，并且更快、更有效地识别路径中断的位置。更深入地分析这个案例将会很有用。添加到操作员工具集中的单个工具极大地缩短了检测故障所需的时间，并且还大大减少了操作员需要交互的设备和命令的数量。应用程序跟踪路由正在利用特定的协议行为[即生存时间/跳数限制的到期、互联网控制消息协议(ICMP)消息的生成]，这可以说是更复杂的协议。最终的结果是操作员接触的设备数量减少到一个。这为复杂性的转变和权衡提供了一个坚实的例子：将复杂性从操作复杂性转变为协议复杂性极大地改善了最终结果。

Alternatively, the operator could use a traceroute from the source node only, and more quickly and efficiently have a potential identification of where the path is broken. It would be useful to analyze this case in some more depth. A single tool added to the operator toolset is dramatically improving the time it takes to detect the failure, and also massively reduces the number of devices and commands that the operator needs to interact with. The application traceroute is leveraging specific protocol behavior [i.e., expiration of the Time-to-Live/Hop Limit, generation of Internet Control Message Protocol (ICMP) messages], which is arguably more protocol complexity. The end result is that the number of devices touched by the operator decreases to one. This makes for a solid example of the shifts in complexity and the tradeoffs: shifting complexity from operational complexity to protocol complexity dramatically improves the end result.

考虑一个稍微更全面的案例来说明类似的结果。假设您正在对损坏的 MPLS 标签交换路径 (LSP) 进行故障排除。操作员可以使用相同的应用程序Traceroute，不仅可以识别问题节点，还可以使用 ICMP MPLS 扩展 (RFC4950) 跟踪整个路径中的 MPLS 标签堆栈。¹然而，在这种情况下，操作员还可以选择使用另一种工具：检测 MPLS 数据平面故障 (RFC4379)。²从协议复杂性的角度来看，这是一种更复杂的方法，但可以提供更细粒度的问题原因识别，并且可以更全面地探索等成本多路径。换句话说，使用 MPLS LSP Ping/Traceroute 这个新工具可以更快、更好地进行故障排除，特别是更全面的路径覆盖和实际问题原因报告。

Consider a slightly more comprehensive case that illustrates similar results. Assume you are troubleshooting a broken MPLS Label Switched Path (LSP). An operator can use the same application, traceroute, and have not only a potential identification of the problem node, but also he or she could trace the MPLS label stack throughout the path with the use of ICMP Extensions for MPLS (RFC4950).¹ In this case, however, the operator could also choose to use another tool: Detecting MPLS Data Plane Failures (RFC4379).² This is a more complex approach from a protocol complexity standpoint, but is one that can provide much more granular problem cause identification, and more comprehensively can explore equal cost multipaths. In other words, using this new tool, MPLS LSP Ping/Traceroute, can result in faster and better troubleshooting, specifically a more comprehensive path coverage and actual problem cause reporting.

1 . Ron Bonica 等人，“多协议标签交换的 ICMP 扩展”（IETF，2007 年 8 月），2015 年 9 月 15 日访问，https ://www.rfc-editor.org/rfc/rfc4950.txt 。

1. Ron Bonica et al., “ICMP Extensions for Multiprotocol Label Switching” (IETF, August 2007), accessed September 15, 2015, https://www.rfc-editor.org/rfc/rfc4950.txt.

2 . K. Kompella 和 G. Swallow，“检测多协议标签交换 (MPLS) 数据平面故障”（IETF，2006 年 2 月），2015 年 9 月 15 日访问，https: //www.rfc-editor.org/rfc/rfc4379 .txt。

2. K. Kompella and G. Swallow, “Detecting Multi-Protocol Label Switched (MPLS) Data Plane Failures” (IETF, February 2006), accessed September 15, 2015, https://www.rfc-editor.org/rfc/rfc4379.txt.

为了最终确定此用例，请继续 MPLS LSP 示例，现在假设网络中部署了 RSVP-TE（节点或路径）保护，并且还存在用于快速故障检测的双向转发检测。这些协议将在第 7 章“协议复杂性”中进行更全面的探讨，但了解它们的操作含义也很有趣。快速故障检测和自动故障修复（或旁路）实际上可以被视为“主动故障排除”，其中没有人为干预导致中断被阻止，并允许操作员有更多时间诊断和修复根本原因。

To finalize this use case, continue with the MPLS LSP example and now assume that there is RSVP-TE (node or path) protection deployed in the network, and there is also bidirectional forwarding detection in place for rapid fault detection. These protocols will be explored more fully in Chapter 7, “Protocol Complexity,” but it is also interesting to see their operational implications. A rapid fault detection and automatic fault remediation (or bypass) can actually be seen as “proactive troubleshooting,” in which no human intervention resulted in an outage prevented and allows more time for an operator to diagnose and fix the root cause.

然而，与复杂世界中的所有事物一样，现实世界中也存在许多权衡。

As with all things in the world of complexity, however, there are a number of tradeoffs in the real world.

• 协议成为必须在网络环境中进行管理的另一个“事物”。回到前面的示例，路由协议无需在网络中的每个路由器上手动配置完整的可达性信息，从而降低了复杂性。另一方面，路由协议在操作方面给网络注入了自身的复杂性，包括策略、聚合、操作和其他问题。

• The protocol becomes another “thing” which must be managed within the context of the network. To return to an earlier example, routing protocols reduce complexity by removing the requirement to manually configure full reachability information on every router in the network. On the other hand, routing protocols inject their own complexity into the network in operational terms, including policy, aggregation, operation, and other issues.

• 该协议实际上是一个“有漏洞的抽象”。例如，虽然Traceroute可以公开从一台主机到另一台主机的网络路径，但它可能不会公开特定应用程序或服务的路径真正采用的路径。例如，出于服务质量的原因，语音流量可能会被定向到与网络管理流量不同的一组链路上。

• The protocol is, in effect, a “leaky abstraction.” While traceroute, for instance, can expose the path through the network from one host to another, it might not expose the path a particular application’s or service’s path will really take. For instance, voice traffic might be directed onto a different set of links than network management traffic for quality of service reasons.

没有流量工程的 Traceroute 失败

值得一提的是，不需要基于策略的路由或流量工程就会导致此处描述的类型问题。例如，假设您从设备运行跟踪路由以发现路径上大延迟的来源。结果看起来像这样：

It’s worth mentioning that it doesn’t take policy-based routing or traffic engineering to cause problems of the type described here. Assume, for instance, that you run a traceroute from a device to discover the source of a large delay along the path. The result looks something like this:

C:\> tracert example.com

1 <1 ms <1 ms <1 ms 192.0.2.45

2 4 ms 3 ms 11 ms 192.0.2.150

3 20 ms 4 ms 3 ms 198.51.100.36

4 * * * 请求超时。

5 * * * 请求超时。

6 7 毫秒 7 毫秒 7 毫秒 203.0.113.49

C:\>tracert example.com

1 <1 ms <1 ms <1 ms 192.0.2.45

2 4 ms 3 ms 11 ms 192.0.2.150

3 20 ms 4 ms 3 ms 198.51.100.36

4 * * * Request timed out.

5 * * * Request timed out.

6 7 ms 7 ms 7 ms 203.0.113.49

这两个“加星号”的啤酒花看起来确实很可疑，不是吗？但实际上，它们可能意味着任何事情，也可能毫无意义。具体有三点值得注意。首先，并非路径上的每个设备都会减少 IP 数据包上的 TTL。例如，作为交换机而不是路由器连接的数据链路层防火墙将在不修改 TTL 的情况下传递流量。其次，隧道可能表现为单跳或多跳。基于外部隧道标头交换数据包的设备（大多数情况下）不会对内部标头 TTL 执行任何操作，这意味着大量设备可能看起来是单个设备。第三，返回路径不显示在traceroute的输出中。——然而，返回路径与出站路径一样与网络性能相关。

The two “starred” hops certainly look suspicious, don’t they? In reality, though, they could mean anything—or nothing. Three specific points are worth noting. First, not every device along a path will decrement the TTL on an IP packet. Data link layer firewalls that connect as switches rather than routers will pass traffic through without modifying the TTL, for instance. Second, tunnels may appear as a single hop or many. A device that switches a packet based on an outer tunnel header will not (most of the time) do anything with the inner header TTL, which means a large set of devices can appear to be a single device. Third, the return path is not shown in the output of a traceroute—and yet the return path has as much to do with network performance as the outbound path.

在所有这三种情况下，以及其他许多情况下，网络都被抽象化，看起来比实际情况更简单。确实存在的跃点不会暴露，并且整个往返路径的一半根本不会显示。然而，这些抽象会泄漏，影响通过网络传输流量的应用程序的性能。缺乏对网络实际运行的可见性可能会对发现和解决问题以及理解网络的实际运行产生重大影响。

In all three of these cases—and many others besides—the network has been abstracted to appear simpler than it actually is. Hops that do exist are not exposed, and half of the entire round trip path is not shown at all. These abstractions leak, however, in impacting the performance of applications carrying traffic over the network. This lack of visibility into the actual operation of the network can have a major impact on finding and resolving problems, as well as in understanding the actual operation of the network.

最后的想法

A Final Thought

本章涵盖了问题空间和操作复杂性问题的一些解决方案。问题空间的两个早期示例是人类与网络交互的成本，以及策略分散与最佳网络使用。这两个问题都说明了一点：网络规模与所提供服务的复杂性的关系与网络物理互连的设备数量的关系一样大。大量设备，每个设备都服务于类似的目的，因此几乎没有配置差异来支持广泛的策略，与承载广泛服务的中等规模的网络相比，具有完全不同的一组扩展问题。这个，如果不出意外的话，

This chapter has covered both the problem space and some solutions for the problem of operational complexity. The two earlier examples of the problem space are the cost of human interaction with the network, and policy dispersion versus optimal network usage. Both of these problems illustrate a single point—the scale of a network relates as much to the complexity of the services offered as it does to the number of devices physically interconnected by the network. A large number of devices, each serving a similar purpose, and therefore with few configuration differences to support a wide array of policies, have a completely different set of scaling issues than a moderately sized network carrying a wide array of services. This, if nothing else, is a useful takeaway from examining management complexity in a network—the problem of scale doesn’t have a single axis, but rather at least two axes.

在检查了一些问题空间之后，本章接着讨论了复杂性问题的一些潜在解决方案，包括自动化、模块化和协议复杂性（或添加额外的协议）。然而，每一种方法都需要权衡，从脆性到状态，再到添加更多必须管理的表面相互作用和系统。在每种情况下，一些复杂性都被“扔到隔间墙上”到其他地方进行管理。当然，自动化是真正大规模网络运行的要求，但重要的是不要将自动化用作复杂系统的一层薄薄的油漆。将复杂性抛到隔间墙上，交给开发运营团队的某人或附近友好的编码员，可能会让网络看起来更简单，

After examining some of the problem space, this chapter then looked at some of the potential solutions to the complexity problems, including automation, modularity, and protocol complexity (or adding additional protocols). Each of these has tradeoffs, however, from brittleness to state to adding more surface interactions and systems that must be managed. In each of these cases, some complexity is being “tossed over the cubicle wall” to some other place to be managed. Automation is, of course, a requirement in the operation of truly large-scale networks—but it’s important not to use automation as a thin coat of paint over what is an otherwise complex system. Tossing complexity over the cubicle wall, to someone on a development operations team, or a nearby friendly coder, might make the network seem simpler, but the complexity required to solve real-world problems doesn’t ever go away.

虽然本章不是本书的重点，但它确实为网络工程师提供了一些需要考虑的思考材料，并为工程师提供了一些寻求复杂性权衡的地方。下一章将从操作复杂性转向设计复杂性。许多相同的问题、解决方案和权衡将会出现，但从不同的角度和不同的领域表达。

While this chapter isn’t a major focus of this book, it does, hopefully, provide some thinking material for network engineers to consider, and some places for engineers to look for complexity tradeoffs. The next chapter moves from operational complexity to design complexity. Many of the same problems, solutions, and tradeoffs will appear, but expressed from a different view—and in a different domain.

第 5 章设计复杂性

Chapter 5. Design Complexity

任何在网络或运营领域工作了很长时间的人都会有一个关于网络拓扑的故事，这简直是令人难以抗拒。就像有一次，一家大公司的网络设计师站在会议室墙上挂着的一百多页的活动挂图前，解释其网络物理层拓扑的细节。或者通过单个园区使用数百条 T1 速度链路构建的网络，因为带宽需求早已超出了并行的 10 或 15 个 T1 的容量，但当地电信公司不会提供任何类型的更大链路。

Anyone who’s been in the network or operations world for a long time has a story about a network topology that was simply overwhelming. Like the time the network designer for one large company stood in front of a flip chart of over a hundred pages hung on the wall of a conference room explaining the ins and outs of his network’s physical layer topology. Or the network that was built with hundreds of T1 speed links through a single campus because the bandwidth requirements had long ago overrun the capacity of ten or fifteen T1s in parallel, but the local telco wouldn’t offer any sort of larger link.

网络设计中的这些大规模失败常常被认为是复杂网络中最糟糕的情况，因为网络复杂性通常等同于拓扑复杂性——有多少链路、它们放置在哪里、有多少环路等。拓扑的网状程度越高，网络就越复杂（如果在孩子的房间里找到路比网络拓扑更容易，或者如果您的网络比孩子的房间更网状，那么您就知道有问题了）。

These large-scale failures in network design are often trotted out as the worst of the worst in complex networks because network complexity is often equated with topological complexity—how many links are there, where are they placed, how many loops, etc. The more highly meshed the topology, the more complex the network (if it’s easier to find your way through your child’s room than your network topology—or if your network is meshier than your child’s room, you know you have a problem).

然而，如果将拓扑复杂性与设计复杂性等同起来，我们就会以一些不太有用的方式限制我们的视野，因为拓扑在复杂性方面并不是孤立的。构建非常密集且具有大量节点的拓扑非常容易，但又不是那么复杂。例如，考虑一个典型的折叠单级脊叶拓扑。有很多连接、很多链路和很多设备，但拓扑相当容易理解，也很容易解释。

Equating topological complexity with design complexity, however, we are limiting our field of view in some very less than useful ways—because the topology doesn’t stand alone in terms of complexity. It’s quite easy to build a topology that is very dense, and has a lot of nodes, and yet isn’t all that complex. Think, for instance, of a typical folded single stage spine-and-leaf topology. There are a lot of connections, a lot of links, and a lot of devices—but the topology is fairly easy to understand, and fairly easy to explain.

那么，难以理解的拓扑为何以及如何与网络设计复杂性相关呢？除了直觉的视觉反应之外，还有什么东西可以将复杂性和意大利面条式的拓扑联系在一起呢？

So why and how does a difficult to understand topology relate to network design complexity? Beyond the gut visual reaction, what else is there that ties complexity and spaghetti-like topologies together?

我们在本章中要考虑的是拓扑与控制平面的关系，因为这可能是更难管理的表面交互之一。交互的两个表面——传输数据包的层（例如物理层或隧道）和控制平面——本身就是非常复杂的系统，每个系统都由许多子系统组成，因此呈现出自己的深层复杂性问题。将两个复杂的系统沿着一个庞大的、定义不明确且经常变化的交互界面组合起来，最终会得到比你想要处理的更复杂的结果。

The one that we’ll consider in this chapter is the topology’s relationship with the control plane, as this is probably one of the more difficult surface interactions to manage. The two surfaces interacting—the layer that is transporting packets (such as the physical layer or a tunnel) and the control plane—are very complex systems on their own, each consisting of a number of subsystems, and hence presenting their own deep complexity problems. Combine two complex systems along a large, ill-defined, and oft-changing interaction surface, and you end up with more complexity than you want to deal with.

控制平面与传输层拓扑交互的三个主要区域如下：

Three primary areas where the control plane interacts with the transport layer’s topology are as follows:

• 控制平面状态的数量。

• The amount of control plane state.

• 控制平面状态变化的速率。

• The rate of control plane state change.

• 拓扑中的任何变化必须进行的传播范围——拓扑变化必须在控制平面内进行多远。

• The scope of propagation through which any change in the topology must be carried—how far a topological change must be carried within the control plane.

回顾第二章“复杂性的组成部分” ，考虑与复杂性的组成部分相关的这三点是很有趣的。

It’s interesting to consider these three points in relation to the components of complexity back in Chapter 2, “Components of Complexity.”

•状态：拓扑如何影响或减少控制平面正在处理的状态量？

• State: How does the topology contribute to, or take away from, the amount of state the control plane is handling?

•速度：拓扑如何影响或降低控制平面对变化做出反应的速度，或者甚至影响向控制平面报告拓扑变化的速率？

• Speed: How does the topology contribute to, or take away from, the speed at which the control plane must react to changes, or even the rate at which changes in the topology are reported to the control plane?

•表面：拓扑变化会影响控制平面中的多少设备？控制平面和数据平面或拓扑之间的交互有多深？控制平面组件有多少个地方交互？

• Surface: How many devices in the control plane does a change in the topology impact? How deep is the interaction between the control plane and the data plane, or the topology? How many places do control plane components interact?

虽然我们可以检查很多不同的点来详细了解这些相互作用，但我们将限制为四个（否则，我们可能会有一整堆书，而不是一个章节）。

While there are a lot of different points we could examine to understand these interactions in some detail, we’ll limit ourselves to four (otherwise, we could have an entire stack of books, rather than a single chapter).

第一个例子是控制平面状态与拉伸——这是一个清楚的例子，说明如何在控制平面承载的信息量以及控制平面的变化率与网络效率之间进行权衡。

The first example is control plane state versus stretch—a clear example of how the amount of information carried by the control plane, and the rate of change in the control plane, can be traded off against the efficiency of the network.

第二个例子是拓扑与收敛速度的关系。我们不仅会在这里研究弹性和冗余之间的关系，还将研究拓扑和弹性之间的关系。

The second example will be topology versus speed of convergence. Not only will we look at the relationship between resilience and redundancy here, we’ll also look at the relationship between topology and resilience.

第三个例子，快速收敛，延续了收敛速度的主题，因此网络性能也是复杂性的一个来源，以及围绕收敛的权衡。

The third example, fast convergence, continues the theme of convergence speed, and hence network performance, as a source of complexity—and the tradeoffs surrounding convergence.

我们的最后一个例子是虚拟化与设计复杂性。对于大多数网络工程师来说，即使经过多年的实践，这一点也不会那么明显，但复杂性与虚拟化网络工程师实践的各种形式之间存在一定的关系。

Our final example will be virtualization versus design complexity. This one won’t be so obvious to most network engineers, even after many years of practice, but there is a definite relationship between complexity and the various forms of virtualization network engineers practice.

控制平面状态与拉伸

Control Plane State versus Stretch

什么是网络延伸？用最简单的术语来说，它是网络中最短路径与两点之间实际路径流量之间的差异。图 5.1说明了这个概念。

What is network stretch? In the simplest terms possible, it is the difference between the shortest path in a network and the path traffic between two points actually takes. Figure 5.1 illustrates this concept.

图 5.1 拉伸，如图所示

Figure 5.1 Stretch, Illustrated

假设该网络中每条链路的成本相同，则路由器 A 和 C 之间的最短物理路径也将是最短逻辑路径：[A，B，C]。但是，如果我们将 [A,B] 链接上的度量更改为 3，会发生什么情况？最短物理路径仍然是 [A,B,C]，但最短逻辑路径现在是 [A,D,E,C]。最短物理路径和最短逻辑路径之间的差异是在路由器 A 和 C 之间转发的数据包必须经过的距离 - 在这种情况下，拉伸可以计算为 (4 [A,D,E,C])– (3 [A,B,C])，拉伸 1。

Assuming that the cost of each link in this network is the same, the shortest physical path between Routers A and C will also be the shortest logical path: [A,B,C]. What happens, however, if we change the metric on the [A,B] link to 3? The shortest physical path is still [A,B,C], but the shortest logical path is now [A,D,E,C]. The differential between the shortest physical path and the shortest logical path is the distance a packet being forwarded between Routers A and C must travel—in this case, the stretch can be calculated as (4 [A,D,E,C])–(3 [A,B,C]), for a stretch of 1.

拉伸度是如何测量的

How is stretch measured

拉伸是否应该根据跳数、指标汇总、网络延迟或其他方式来衡量？这取决于在任何给定情况下什么是最重要的，但最常见的方法是通过网络比较跳数，为简单起见，如此处示例中所使用的那样。在某些情况下，考虑沿两条路径的度量、沿两条路径的延迟或其他一些度量可能更重要，但重要的一点是在每个可能的路径上一致地测量它，以便在路径之间进行准确比较。

Should stretch be measured in terms of hop count, the summary of the metrics, the delay through the network, or some other way? It depends on what is most important in any given situation, but the most common way is by comparing hop counts through the network, as is used in the examples here for simplicity. In some cases, it might be more important to consider the metric along two paths, the delay along two paths, or some other metric, but the important point is to measure it consistently across every possible path to allow for accurate comparison between paths.

关于拉伸有几点值得注意：

Several points are worth noting about stretch:

• 有时很难区分物理拓扑和逻辑拓扑。在这种情况下，[A,B] 链路指标是否增加是因为该链路实际上是较慢的链路？如果是这样，那么这是否是拉伸的示例，还是简单地使逻辑拓扑与物理拓扑保持一致的示例是值得商榷的。

• It’s sometimes difficult to differentiate between the physical topology and the logical topology. In this case, was the [A,B] link metric increased because the link is actually a slower link? If so, whether this is an example of stretch, or an example of simply bringing the logical topology in line with the physical topology is debatable.

• 根据这一观察结果，用弹性来定义政策比几乎任何其他方式都要容易得多。策略是任何增加网络延伸的配置。例如，使用基于策略的路由或流量工程将流量从最短的物理路径推送到较长的逻辑路径以减少特定链路上的拥塞，这是一种策略，它可以增加伸展度。

• In line with this observation, it’s much easier to define policy in terms of stretch than almost any other way. Policy is any configuration that increases the stretch of a network. Using Policy Based Routing, or Traffic Engineering, to push traffic off the shortest physical path and onto a longer logical path to reduce congestion on specific links, for instance, is a policy—it increases stretch.

• 增加伸展力并不总是坏事。理解拉伸的概念可以帮助我们理解各种其他概念，并围绕复杂性权衡建立一个框架。从物理角度来说，最短路径并不总是最佳路径。

• Increasing stretch is not always a bad thing. Understanding the concept of stretch simply helps us understand various other concepts, and put a framework around complexity tradeoffs. The shortest path, physically speaking, isn’t always the best path.

• 在此图中，拉伸非常简单——它会影响每个目的地以及流经网络的每个数据包。在现实世界中，事情并不那么简单。拉伸实际上是按源/目标对进行的，因此很难在网络范围内进行测量。

• Stretch, in this illustration, is very simple—it impacts every destination, and every packet flowing through the network. In the real world, things aren’t so simple. Stretch is actually per source/destination pair, making it very difficult to measure on a network wide basis.

考虑到所有这些，让我们看一下拉伸和优化之间权衡的两个具体示例。

With all of this in mind, let’s look at two specific examples of the tradeoff between stretch and optimization.

聚合与拉伸

Aggregation versus Stretch

聚合是一种不仅可以减少控制平面中承载的信息量，还可以减少控制平面中的状态变化率的技术。聚合内置于 IP（IPv4 和 IPv6）中，即使单个子网也包含多个主机地址。通过将单个广播段连接到一组主机，IP 路由协议不需要管理第 2 层可达性，也不需要管理单个主机地址。

Aggregation is a technique used to reduce not only the amount of information carried in the control plane, but also the rate of state change in the control plane. Aggregation is built into IP (both IPv4 and IPv6)—even a single subnet contains multiple host addresses. By connecting a single broadcast segment to a set of hosts, the IP routing protocol doesn’t need to manage Layer 2 reachability, nor individual host addresses.

笔记

Note

主机地址有时用于在较大的网络中提供移动性，特别是数据中心结构和移动自组织网络 - 这是我们在此不考虑的状态与延伸权衡的说明。

Host addresses are sometimes used to provide mobility within a larger network, particularly data center fabrics and mobile ad-hoc networks—an illustration of the state versus stretch tradeoff we won’t consider here.

控制平面内的聚合也会导致拉伸，如图5.2所示。

Aggregation within the control plane can also cause stretch, as Figure 5.2 shows.

图 5.2 聚合和拉伸

Figure 5.2 Aggregation and Stretch

笔记

Note

大多数路由协议都会考虑聚合组件的度量，以在此类聚合边缘提供尽可能接近最佳的路由。然而，即使采取了这些应对措施，将拉伸引入网络仍然很常见。该示例无需考虑组件度量，以使原理更容易说明。

Most routing protocols take into account the metrics of an aggregate’s components to provide as close to optimal routing as possible at an aggregation edge of this type. However, it’s still common for stretch to be introduced into the network even with these counter measures. This example dispenses with taking component metric into account to make the principle easier to illustrate.

两种不同的情况说明了通过路由聚合增加延伸：

Two different situations illustrate increasing stretch through route aggregation:

• 假设[A,B] 链路的成本为2，该网络中所有其他链路的成本为1。如果路由器B 和C 都聚合到2001:db8::/61，则通过[ A,C] 对于聚合中的所有内容都是首选。发往 2001:db8:0:1::/64 的流量将沿着路径 [A,C,E,D] 到达目的地，即使最短（物理）路径是 [A,B,D]。2001:db8:0:2::/64 的拉伸不变，但 2001:db8:0:1::/64 的拉伸增加了 1。

• Assume the [A,B] link has a cost of 2, and all the other links in this network have a cost of 1. If Routers B and C both aggregate to 2001:db8::/61, then the path through [A,C] would be preferred for everything within the aggregate. Traffic destined to 2001:db8:0:1::/64 will pass along the path [A,C,E,D] to reach its destination, even though the shortest (physical) path is [A,B,D]. The stretch for 2001:db8:0:2::/64 isn’t changed, but the stretch for 2001:db8:0:1::/64 is increased by one.

• 假设网络中所有链路的成本均为 1。如果路由器 B 和 C 都聚合到 2001:db8::/61，则路由器 A 将以某种方式将流量负载共享到路由器 D 和 E 后面的两个子网。它具有可用的等价路径。假设完美的负载共享，50% 发往 2001:db8:0:1::/64 的流量将沿 [A,C,E,D] 流动，延伸段为 1，50% 发往 2001:db8:0:1::/64 的流量将沿 [A,C,E,D] 流动2001:db8:0:2::/64 将沿 [A,B,D,E] 流动，拉伸长度为 1。

• Assume all the links in the network have a cost of 1. If Routers B and C both aggregate to 2001:db8::/61, then Router A will somehow load share traffic toward the two subnets behind Routers D and E across the two equal cost paths it has available. Given perfect load sharing, 50% of the traffic destined to 2001:db8:0:1::/64 will flow along [A,C,E,D], with a stretch of 1, and 50% of the traffic destined to 2001:db8:0:2::/64 will flow along [A,B,D,E], with a stretch of 1.

现在应该想到的问题是——那又怎样？通过聚合（状态的实际数量和该状态的变化率）减少了控制平面状态的数量，这明显降低了复杂性。这里需要权衡三种结果：

The question that should come to mind about now is this—so what? The amount of control plane state is decreased through aggregation (both the actual amount of state and the rate of change in that state), which clearly reduces complexity. There are three outcomes being traded off here:

• 实施聚合可以打破故障域，提高控制平面（进而提高整个网络）的稳定性和弹性。

• Implementing aggregation breaks up failure domains, improving the stability and resilience of the control plane (and hence of the entire network).

• 实施聚合需要设计、配置和维护围绕聚合本身的策略集。这是额外的复杂性。

• Implementing aggregation requires designing, configuring, and maintaining the set of policies around the aggregation itself. This is additional complexity.

• 在此网络中实施聚合会导致通常的两条冗余链路变成单点故障。如果Router B是到达聚合组2001:0db8::/61的首选路由，并且Router B之间的链路路由器 D 和 E 发生故障，2001:0db8:0001::/64 将无法从路由器 A 到达。发往此前缀的流量将转发到路由器 C，因为聚合覆盖了此地址空间，然后被丢弃，因为路由器 C 将有没有到特定子网的路由。为了解决这个问题，必须安装一条新路径，提供从路由器 C 到路由器 D 的备用路径。通常，该链路将安装在路由器 B 和 C 之间，并配置为位于聚合“后面”，而不是“前面” of”聚合（从路由器 C 的角度来看，聚合必须配置到路由器 A，而不是路由器 B）。解决聚合黑洞问题会增加多个领域的复杂性。

• Implementing aggregation in this network causes what would normally be two redundant links to become a single point of failure. If Router B is the preferred route to the aggregate 2001:0db8::/61, and the link between Routers D and E fails, 2001:0db8:0001::/64 will become unreachable from Router A. Traffic destined to this prefix will be forwarded to Router C, because the aggregate covers this address space, and then dropped because Router C will have no route to the specific subnet. To remedy this, a new path must be installed providing an alternate path to Router D from Router C. Normally, this link would be installed between Routers B and C, and configured so it is “behind” the aggregation, rather than “in front of” the aggregation (aggregation must be configured toward Router A from Router C’s perspective, not Router B). Solving this aggregation black hole problem increases complexity in several areas.

• 增加延伸会增加数据平面的复杂性，因为流量会经过更多的跃点，从而增加队列等。

• Increasing stretch increases the complexity of the data plane by pushing traffic through more hops, and hence more queues, etc.

• 增加拉伸会断开网络明显/表观的操作与网络实际工作的方式。随着增加延伸的策略的实施，查看网络拓扑并了解流量如何流经网络变得更加困难。这增加了对网络操作中可能（事实上，几乎肯定会）出现的各种问题进行故障排除的复杂性和难度。

• Increasing stretch disconnects the obvious/apparent operation of the network from the way the network actually works. As policies are implemented that increase stretch, it becomes more difficult to look at the network topology and understand how traffic flows through the network. This increases the complexity and difficulty of troubleshooting various problems that might (in fact, almost certainly will) arise in operating the network.

• 增加延伸可提高网络的整体利用率，而不会实际增加通过网络传输的流量。在图 5.1给出的示例中，通常采用两跳路径的流量被定向为三跳路径，这意味着转发和交换流中的数据包涉及多一条链路和多一台路由器通过网络。从纯粹的数学角度来看，增加拉伸会增加用于转发任何特定流的设备和链路的数量，从而降低网络的整体效率。

• Increasing stretch increases the overall utilization of the network without any actual increase in the amount of traffic being carried through the network. In the example given in Figure 5.1, traffic that would normally take a two-hop path is directed along a three-hop path, which means one more link and one more router are involved in forwarding and switching the packets in the flow(s) across the network. In purely mathematical terms, increasing stretch decreases the overall efficiency of the network by increasing the number of devices and links used to forward any particular flow.

最后一点很有趣，因为它切入了我们下一次拉伸与控制平面状态比较的核心，即流量工程如何影响复杂性。

This last bullet is an interesting point—for it cuts to the heart of our next comparison of stretch versus control plane state, how traffic engineering impacts complexity.

交通工程

Traffic Engineering

让我们看一个更具体的流量工程状态交易示例。我们将使用图 5.3中的网络作为示例。

Let’s look at a more specific example of trading state for traffic engineering. We’ll use the network in Figure 5.3 as an example.

图 5.3 流量工程、状态和延伸

Figure 5.3 Traffic Engineering, State, and Stretch

假设该网络中每条链路的成本为1；2001:db8:0:1::/64 和 2001:db8:0:2::/64 之间的最短路径是 [A,B,F]。但是，那网络管理员不希望这两个子网之间的流量沿着该路径传输，例如，正在交换的流量可能是高带宽文件传输流，并且 [A,B] 链路已用于视频流。为了防止两个流“冲突”，从而导致服务质量问题，网络管理员在路由器 A 上实施某种形式的基于策略的路由，通过路径 [A,C,E,F] 重定向文件传输流量。

Assume that every link in this network has a cost of 1; the shortest path between 2001:db8:0:1::/64 and 2001:db8:0:2::/64 is [A,B,F]. However, the network administrator doesn’t want traffic between these two subnets traveling along that path—for instance, perhaps the traffic being exchanged is a high bandwidth file transfer stream, and the [A,B] link is already used for a video stream. To prevent the two streams from “colliding,” causing quality of service issues, the network administrator implements some form of policy-based routing at Router A that redirects the file transfer traffic through the path [A,C,E,F].

考虑一下这涉及到什么：

Consider what this involves:

• 现在必须在路由器A 和F 上创建并安装策略，以重定向[A,B,F] 路径周围的特定流量。

• A policy must now be created and installed on Routers A and F to redirect this specific traffic around the [A,B,F] path.

• 文件传输流量的扩展已增加一。

• The stretch for the file transfer traffic has been increased by one.

• 这种额外的延伸意味着流量现在将经过四跳，而不是三跳，这意味着必须管理四个输入队列、四个输出队列、四个转发表等，而不是三个。

• This additional stretch means the traffic is now going to pass through four hops, rather than three, which means four input queues, four output queues, four forwarding tables, etc., must be managed, instead of three.

• 从拓扑上看，任何给定流通过网络的路径不再明显；为了了解特定流的路径，网络工程师解决问题必须检查控制平面策略，以确定每种特定情况下会发生什么。

• The path of any given stream through the network is no longer obvious from the topology; to understand the path of a specific stream, a network engineer troubleshooting problems must examine the control plane policies to determine what will happen in each specific case.

每一个都增加了一点复杂性——事实上，通过这个练习，数据平面和控制平面的复杂性都增加了。那么我们得到了什么？为什么要做这样的交通工程？

Each of these is an added bit of complexity—in fact, both the data plane and the control plane complexity have increased through this exercise. So what have we gained? Why do this sort of traffic engineering?

因为即使网络的整体效率（就利用率而言）因沿着具有一定延伸的路径重定向某些流量而降低，但网络的整体利用率在其处理负载的能力方面有所增加，或者更确切地说是支持一组特定的应用程序。为了把这个换句话说，尽管整体效率较低，但网络现在的利用率较高。这可以防止网络运营商被迫安装新链路或升级容量——从运营商的角度来看，这都是好事。

Because even though the overall efficiency of the network (in terms of utilization) has decreased by redirecting some traffic along a path with some stretch, the overall utilization rate of the network has increased in terms of its ability to handle load, or rather to support a specific set of applications. To put this in other terms, the network now has a higher utilization rate even though the overall efficiency is lower. This prevents the network operator from being forced to install new links, or upgrade capacity—both good things from the operator’s point of view.

状态与拉伸：一些最终想法

State versus Stretch: Some Final Thoughts

因此，在状态与拉伸的情况下，每当我们增加拉伸时，我们都在实施某种策略，而每当我们实施策略时，我们都在以某种方式增加复杂性。另一方面，为了减少控制平面中的状态或提高整体网络利用率，通常需要创建延伸策略。

So in the case of state versus stretch, any time we are increasing stretch, we are implementing some sort of policy, and any time we are implementing a policy, we are increasing complexity in some way. On the other hand, creating stretch through policy is often necessary to either reduce state in the control plane, or to improve overall network utilization.

那么，状态与伸展是一组权衡。“我应该实施这项增加弹性的政策吗？”这个问题没有绝对正确的答案。正确的答案始终取决于目标和预计的后果。在我们进入下一节考虑拓扑设计和收敛之间的权衡之前，将状态与拉伸联系回第 2 章“复杂性的组成部分”中概述的状态、速度、表面框架非常重要：

State versus stretch is, then, a set of tradeoffs. There is no absolutely right answer to the question, “Should I implement this policy that increases stretch?” The right answer is always going to depend on the goals and the projected consequences. Before we jump into the next section considering the tradeoff between topology design and convergence, it’s important to tie state versus stretch back to the state, speed, surface framework outlined in Chapter 2, “Components of Complexity”:

•状态：控制平面中的信息量或者通过聚合减少，或者通过流量工程增加。在一种情况下，控制平面的复杂性正在降低，而网络复杂性（在数据平面中）却在增加，至少在一定程度上是增加的。

• State: The amount of information in the control plane is either decreased through aggregation, or increased through traffic engineering. In the one case, the amount of control plane complexity is decreasing, while the network complexity (in the data plane) is increasing, at least in some amount.

•速度：聚合通过隐藏网络中的变化来降低控制平面必须做出反应的速度。第 6 章“管理设计复杂性”更详细地讨论了这一点。向控制平面添加更多状态来设计流量会产生与聚合相反的效果；单个链路故障可能意味着控制平面中的多次更新，因此控制平面必须更快地做出反应。

• Speed: Aggregation reduces the speed at which the control plane must react by hiding changes in the network. Chapter 6, “Managing Design Complexity,” looks at this in more detail. Adding more state to the control plane to engineer traffic has an effect that’s the opposite of aggregation; a single link failure can mean multiple updates in the control plane, so the control plane must react more quickly.

•表面：聚合以某些方式减少了控制平面和拓扑之间的交互表面（通过对控制平面隐藏拓扑的某些部分），并以其他方式增加了交互表面（通过实施必须与特定的控制平面策略一致）网络内的拓扑特征。流量工程增加了控制平面和拓扑之间的交互面。

• Surface: Aggregation reduces the interaction surface between the control plane and the topology in some ways—by hiding some parts of the topology from the control plane—and increases the interaction surface in other ways—by implementing control plane policy that must coincide with specific topological features within the network. Traffic engineering increases the interaction surface between the control plane and topology.

事实证明，状态、速度和表面框架对于诊断复杂性增加、复杂性降低以及权衡所在的各个点非常有用。

The state, speed, and surface framework proves useful in diagnosing the various points where complexity has increased, where complexity has decreased, and where the tradeoffs lie.

拓扑与收敛速度

Topology versus Speed of Convergence

延伸并不是网络拓扑和控制平面相互交互的唯一地方。拓扑的实际布局也是一个（通常不那么明显）交互点。让我们考虑拓扑和控制平面之间交互的两个具体示例：环形拓扑收敛以及冗余与弹性。

Stretch isn’t the only place where the network topology and the control plane interact with one another. The actual layout of the topology is a (often not so obvious) point of interaction, as well. Let’s consider two specific examples of interactions between the topology and the control plane: ring topology convergence, and redundancy versus resilience.

环形拓扑收敛

Ring Topology Convergence

环形拓扑（如图5.4所示）具有一组相当具体的收敛特性。

Ring topologies, such as the one illustrated in Figure 5.4, have a fairly specific set of convergence characteristics.

图 5.4 环形拓扑收敛

Figure 5.4 Ring Topology Convergence

如果[D,2001:db8:0:1::/64]链路出现故障，该拓扑上距离矢量协议的收敛过程如何？

If the [D, 2001:db8:0:1::/64] link fails, what is the convergence process for a distance vector protocol on this topology?

1. RouterD发现故障。

1. Router D discovers the failure.

2. RouterD将故障通告给RouterC和E。

2. Router D advertises the failure to Routers C and E.

3.路由器 C 和 E 向 B 和 F 通告 2001:db8:0:1::/64 的链路丢失。

3. Routers C and E advertise the loss of the link to 2001:db8:0:1::/64 to B and F.

4.路由器B和F向路由器A通告2001:db8:0:1::/64的链路丢失。

4. Routers B and F advertise the loss of the link to 2001:db8:0:1::/64 to Router A.

5.到 2001:db8:0:1::/64 的路由将从所有本地路由表中删除，并丢弃发往该子网的流量。

5. The route to 2001:db8:0:1::/64 is removed from all the local routing tables, and traffic destined to this subnet is dropped.

链路状态协议怎么样？

What about with a link state protocol?

1. RouterD发现故障。

1. Router D discovers the failure.

2.路由器 D 将修改后的链路状态信息洪泛到网络中（或至少在洪泛域内）的其余路由器。

2. Router D floods modified link state information to the rest of the routers in the network (or at least within the flooding domain).

3.然后，每个路由器设置一个定时器来延迟运行 SPF。

3. Each router then sets a timer to delay running SPF.

4.当每个路由器运行SPF时，它将重新计算到2001:db8:0:1::/64的最佳路径并安装它。

4. As each router runs SPF, it will recalculate the best path to 2001:db8:0:1::/64 and install it.

5.由于环内的路由器在不同的时间运行SPF，它们会在不同的时间安装通往该目的地的新路径，从而导致微环，而整个环的路由表处于不一致的状态。（请注意，在本例中，删除路线后不会发生这种情况。）

5. As routers in the ring run SPF at different times, they will install the new path toward this destination at different times, resulting in microloops while the routing tables throughout the ring are in an inconsistent state. (Note this won’t happen with a route removal, as in this case.)

6.经过一段时间后，所有路由器都运行了 SPF，重新计算了到达目的地的最佳路径，并安装了路由。

6. After some period of time, all routers have run SPF, recalculated their best paths to the destination, and installed the routes.

因此，无论您使用哪种类型的路由协议，当有关特定目的地的更改的信息通过环传播并起作用时，都会存在一些时间滞后。您基本上可以在收敛期间选择丢弃或循环流量。

So no matter which type of routing protocol you’re using, there is some time lag as the information about changes to specific destination is propagated and acted on through the ring. You essentially have your choice between dropping or looping traffic during convergence.

那么，为什么网络设计者经常使用环呢？主要原因如下：

Why, then, do network designers often use rings? The primary reasons are as follows:

• 环是连接最少的丰富拓扑，提供辅助路径（更正式的术语是两个连通图）。

• Rings are the least connection rich topology that offers a secondary path (a two connected graph, in more formal terms).

• 由于环的连接并不丰富，因此在环拓扑上不存在邻居数量等的扩展问题——无论环发展到多大，每个路由器都只会有两个邻居。

• Because rings are not connection rich, there are no scaling issues with the number of neighbors, etc., on a ring topology—no matter how large the ring grows, each router will only have two neighbors.

• 环适用于通过最少的（昂贵的）链路跨越较长的地理距离。

• Rings are good for spanning long geographic distances with a minimal set of (expensive) links.

• 就复杂性而言，环形拓扑为网络（路由或IP）控制平面添加的负载要少得多。

• In terms of complexity, ring topologies add a lot less load to the network’s (routed or IP) control plane.

三角形（具有三跳的较大拓扑的任何部分）比环（具有四跳或更多跳的拓扑的任何部分）收敛得快得多，但三角形在控制平面上施加了更多负载 - 特别是在信息量方面每个节点接收。那么，到这里，我们就清楚了拓扑、控制平面和收敛速度之间相互作用的复杂性权衡。

Triangles (any part of a larger topology with three hops) converge much faster than rings (any part of a topology with four or more hops), but triangles place a lot more load on the control plane—particularly in terms of the amount of information each node receives. Here, then, we have a clear complexity tradeoff in the interaction between topologies, control planes, and convergence speed.

让我们看看网络拓扑图谱的另一端，并以非常密集连接的拓扑为例。

Let’s go to the opposite end of the network topology spectrum and example very densely connected topologies.

冗余与弹性

Redundancy versus Resilience

假设您需要构建一个具有六个 9 可靠性（99.9999% 正常运行时间）的网络，并且您拥有的唯一可用链路在任何给定年份的平均停机时间约为 3.5 天（99% 可靠性）。用低可靠性部件提供高可用性的简单解决方案是将它们中的许多部件并联。表5.1给出了增加并机数量与系统可靠性之间的关系。

Assume that you need to build a network with six nines of reliability (99.9999% uptime), and the only links you have available have an average downtime of about 3.5 days in any given year (99% reliability). The easy solution to providing high availability with low reliability parts is to put many of them in parallel. Table 5.1 shows the relationship between increasing the number of parallel units and the reliability of the system.

表 5.1 可用性与冗余

Table 5.1 Availability versus Redundancy

计算可用性

Calculating Availability

我们是如何找到这些数字的？如果您有每个组件的可用性编号，则并联和串联可用性的公式很简单。对于系列组件，只需将每个系列组件的可用性相乘即可。例如，要查找背靠背连接在一起的两个 99% 可用链接的可用性，请将两个可用性相乘：

How did we find these numbers? The formula for parallel and series availability is simple if you have availability numbers for each component. For series components, simply multiply the availability for each series component together. For instance, to find the availability of two 99% available links tied together back to back, multiply the two availabilities together:

A == A ₁ * A ₂

A == 0.99 * 0.99

A == 0.9801 == 98% 可用

A == A₁ * A₂

A == 0.99 * 0.99

A == 0.9801 == 98% available

对于并联组件，请使用以下公式：

For components in parallel, use the following formula:

A == 1 – ((1 – A ₁ ) * (1 – A ₂ ))

A == 1 – ((1 – 0.99) * (1 – 0.99)

A == 1 – (0.01 * 0.01)

A == 1 – 0.0001

A == 0.9999 == 99.99% 可用

A == 1 – ((1 – A₁) * (1 – A₂))

A == 1 – ((1 – 0.99) * (1 – 0.99)

A == 1 – (0.01 * 0.01)

A == 1 – 0.0001

A == 0.9999 == 99.99% available

如果您要组合并联（并排）和串联（背靠背）组件，则从并行或串联的最小分组开始分段计算，并且当您用较大的部分替换较小的部分时将一组组件视为单个组件。图 5.5用于说明计算具有并联和串联组件的系统的可用性。

If you’re combining parallel (side-by-side) and series (back-to-back) components, then compute in pieces, starting from the smallest groupings either in parallel or series, and, as you replace the smaller sections with larger ones, treating a group of components as a single component. Figure 5.5 is used to illustrate computing the availability of a system with both parallel and series components.

图 5.5 计算并行和串行可用性

Figure 5.5 Computing Parallel and Series Availability

如果路由器 A、B、C 和 D 的单独可用性均为 99%，则可以使用以下公式计算所有四个路由器的可用性（忽略链路）：

If Routers A, B, C, and D all had individual availability rates of 99%, you would compute the availability of all four routers (ignoring the links) using the following formula:

(A,B) == 1 – ((1 – A) * (1 – B))

(A,B) == 1 – ((1 – 0.99) * (1 – 0.99)

(A,B) == 1 – (0.01 * 0.01)

(A,B) == 1 – 0.0001

(A,B) == 0.9999 == 99.99% 可用

(A,B) == 1 – ((1 – A) * (1 – B))

(A,B) == 1 – ((1 – 0.99) * (1 – 0.99)

(A,B) == 1 – (0.01 * 0.01)

(A,B) == 1 – 0.0001

(A,B) == 0.9999 == 99.99% available

鉴于一对路由器 (A,B) 的可用性将与一对路由器 (C,D) 的可用性相同，您现在可以通过计算 (A,B) 的可用性来计算四个路由器的可用性,B) 与 (C,D) 串联：

Given the availability of the pair of routers (A,B) is going to be the same as the availability of the pair of routers (C,D), you can now compute the availability of the four routers by computing the availability of (A,B) in serial with (C,D):

可用性 == (A,B) * (C,D)

A == 0.9999 * 0.9999

A == 0.9998 == 99.98% 可用

Availability == (A,B) * (C,D)

A == 0.9999 * 0.9999

A == 0.9998 == 99.98% available

因此，组合系统（忽略链接）将具有三个九的可用性。

So the combined systems (ignoring the links) would have three nines of availability.

很简单——只需确保网络中任何地方至少有三个并行链路，并且您就有六个九的可靠性，对吧？也许。让我们为路由器添加冗余，然后尝试用它构建一些东西。图 5.6说明了三重冗余全网状拓扑。

Easy—just make certain that there are at least three links in parallel anyplace in the network, and you have six nines of reliability, right? Well, maybe. Let’s add in redundancy for our routers, and then try to build something out of it. Figure 5.6 illustrates a triple redundant full mesh topology.

图 5.6 三重冗余全网状拓扑

Figure 5.6 A Triple Redundant Full Mesh Topology

从链路和路由器的角度来看，这种设计确实满足了六个九的要求，但是控制平面的收敛速度有多快？它甚至可能在“理想”条件下工作，但是当一个链路以导致快速抖动的方式发生故障时，这种设计就会产生有趣的问题。在 EIGRP 上进行的实验室测试中，该协议实际上开始在大约四个并行链路（在具有数万条路由的非常简单的拓扑上）处失去收敛速度。

This design certainly meets the six nines requirement from the perspective of links and routers—but how fast is the control plane going to converge? It might even work under “ideal” conditions, but when one link fails in a way that causes a rapid flap, interesting problems are going to result from this sort of design. In lab testing done on EIGRP, the protocol actually started losing convergence speed at around four parallel links (on a very simple topology with tens of thousands of routes).

网络中泛滥的控制平面流量数量可能会阻碍链路状态协议收敛。例如，如果 (A,2001:db8:0:1::/64) 链路失败，则修改后的链路状态更新将在整个三重全网格中多次泛洪，除非某种泛洪采用减速机制。事实上，这就是 IS-IS 和 OSPF 中存在网状组的原因，以防止全网状网络上发生大规模洪泛事件。除此之外，在此拓扑上收敛期间创建的微环的数量和位置可能很大且难以预测。

Link state protocol convergence could be hampered by the number of control plane traffic flooded through the network. For instance, if the (A,2001:db8:0:1::/64) link fails, the modified link state update will be flooded throughout the triple full mesh multiple times, unless some sort of flooding reduction mechanism is used. This is, in fact, why mesh groups exist in IS-IS and OSPF—to prevent large flooding events on full mesh networks. Beyond this, the number and location of microloops created during convergence on this topology could be large and difficult to predict.

降低此类拓扑复杂性的一种方法，同时保持三重冗余以实现弹性，可能是将路由器之间的每组链路视为单个链路，这可以使用链路聚合组 (LAG) 等技术来实现。），例如。细心的读者已经意识到这里也存在权衡。首先，LAG有自己的控制平面，并对转发平面进行修改，以确保每个并行电路的平等使用。它们将与覆盖的 IP 路由协议交互，创建另一个交互表面。管理此交互表面的困难的一个例子是决定如何处理任意两个路由器之间的三个可用链路中的单个链路的故障。是否应该在路由协议中修改单个聚合链路上的度量？还是应该以同等成本使用该链接？从交通流和负载的角度来看，每种解决方案有何影响？一如既往，这些问题都不容易回答——虽然这样的解决方案可能很适合特定情况，但在复杂性和网络流量优化方面总是需要权衡。

One way to lower the complexity of this type of topology, while maintaining triple redundancy for resilience, might be to treat each of the sets of links between the routers as a single link—this would be possible using a technology like Link Aggregation Groups (LAGs), for instance. Observant readers are already going to realize that there are tradeoffs here, as well. First, the LAG has its own control plane, and makes modifications to the forwarding plane, to ensure equal use of each of the parallel circuits. These will interact with the overlaying IP routing protocol, creating another interaction surface. One example of the difficulty in managing this interaction surface is deciding how to handle the failure of a single link out of the three available between any two routers. Should the metric on the single aggregated link be modified in the routing protocol? Or should the link be used at equal cost anyway? What are the implications of each solution from a traffic flow and loading perspective? These are, as always, not easy questions to answer—while such a solution might fit a particular situation well, then, there are always tradeoffs to consider in terms of complexity and optimization of traffic flow through the network.

因此，这取决于很多因素，但是如果没有对计时器进行认真的调整，并且在此过程中可能会减少一些控制平面状态，那么这里的路由协议可能不会在所有可能的条件下收敛到六个九的要求。

So it depends on a lot of factors, but the routing protocol here is probably not going to converge within the six nines requirement in all possible conditions without some serious tuning of timers, and potentially some reduction of control plane state along the way.

这里的权衡在于冗余和弹性之间——虽然添加冗余链路似乎总是会提高网络弹性，但现实世界中的情况并非如此。每个冗余链路都会增加一点控制平面状态，并在收敛期间增加一点负载。每增加一点负载都会减慢控制平面的收敛速度，在某些情况下甚至会导致控制平面无法收敛。

The tradeoff here is between redundancy and resilience—while it might seem that adding redundant links will always increase network resilience, this simply isn’t the case in the real world. Each redundant link adds a bit more control plane state, and a bit more load during convergence. Each added bit of load slows down the convergence speed of the control plane, and, in some cases, can even cause the control plane to fail to converge.

拓扑与收敛速度：一些最终的想法

Topology versus Speed of Convergence: Some Final Thoughts

让我们回到我们的状态、速度和表面模型，看看拓扑与速度权衡如何以及在何处适合。此处它们的呈现顺序略有不同。

Let’s go back to our state, speed, and surface model to see how and where the topology versus speed tradeoff fits. They will be presented in a slightly different order here.

•状态：随着网络中内置冗余的增加，链路数量和邻接/对等会话数量必须增加，才能发现额外的链路和跨这些链路的邻居关系。这反过来又增加了控制平面中承载的信息量以及整个网络中该数据的复制数量。

• State: As the redundancy built into the network increases, the number of links and the number of adjacencies/peering sessions must increase to discover the additional links and neighbor relationships across those links. This, in turn, increases both the amount of information carried in the control plane and the number of replications of that data throughout the network.

•表面：随着网络中设备数量的增加，参与控制平面的设备之间的邻接或对等关系的数量也必须增加。对等关系的增加代表交互面的增加。任何特定的信息都必须通过更多的设备来承载。

• Surface: As the number of devices in the network increases, the number of adjacencies or peering relationships among devices participating in the control plane must also increase. The increase in peering relationships represents an increase in interaction surfaces. Any particular piece of information must be carried through more devices.

•速度：随着网络内置冗余的增加，有关网络状态的任何特定信息必须复制的次数也会增加。如果拓扑中的任何变化需要控制平面收敛，以便内部数据库的状态与现实世界的状态相匹配（用拓扑术语来说），则总状态中包含的每条附加信息要么需要收敛更长的时间，要么在相同的时间内要分发和计算更多的状态。如果我们期望网络在相同的时间内或更快地收敛更多信息，那么我们期望控制平面能够更快地处理信息。在相同的时间内（或更快）处理的较大数据库需要更高的速度或每秒更多的事务。

• Speed: As the redundancy built into the network increases, the number of times any particular piece of information about the state of the network must be replicated increases. If any change in the topology requires the control plane to converge so the state of the internal databases matches the state of the real world (in topological terms), each additional piece of information included in the total state either requires convergence to take longer, or a larger amount of state to be distributed and calculated across in the same amount of time. If we expect the network to converge in the same amount of time or faster with more information, then we are expecting the control plane to process information more quickly. Larger databases processed in the same amount of time (or faster) require more speed, or more transactions per second.

关于速度的最后一点将作为下一节研究速度和复杂性之间关系的延续——快速收敛。

This last point on speed will operate as a segue into the next section investigating the relationship of speed and complexity—fast convergence.

快速收敛与复杂性

Fast Convergence versus Complexity

当网络技术首次普及时，快速融合意味着速度足够快，可以允许文件传输或电子邮件通过网络而不会造成太多中断。随着网络变得越来越快，部署在网络之上的应用程序是围绕网络可以提供的更快的速度和更高的可用性而构建的。曾经可接受的收敛速度现在对于许多应用来说被认为太慢。考虑下图，显示了最初部署时各种路由协议的理论收敛时间，使用默认计时器并且没有“技巧”，例如无环路替代（LFA）或快速重新路由机制。图 5.7说明了使用默认计时器和“无技巧”的路由协议收敛速度。

When networking technology first became widespread, fast convergence meant fast enough to allow a file transfer or email to make its way through the network without too much interruption. As networks have become faster, the applications deployed on top of them have been built around the faster speeds and higher availability networks can deliver. Once acceptable convergence speeds are now considered too slow for many applications. Consider the following illustration showing the theoretical convergence times of various routing protocols as they were originally deployed, using default timers and no “tricks,” such as loop-free alternates (LFAs) or fast reroute mechanisms. Figure 5.7 illustrates routing protocol convergence speeds with default timers and “No Tricks.”

图 5.7 使用默认计时器和“无技巧”的路由协议收敛速度

Figure 5.7 Routing Protocol Convergence Speeds with Default Timers and “No Tricks”

让我们看看每个协议，以了解此处显示的收敛时间。

Let’s look at each protocol to get a sense for the convergence times shown here.

• 路由信息协议(RIP) 使用定期更新来通知邻居可达性的变化（RIP 并不真正携带拓扑信息，尽管可以从数据库推断出一些有关拓扑的信息）。此定期更新的“默认”计时器为 30 秒；平均（统计），任何给定的可达性变化都会等待 15 秒，然后才会被通告给相邻路由器。那么，通知整个网络可达性变化的平均时间就是可达性变化必须经过的最大跳数乘以 15 秒。对于大型网络，这可能意味着两到三分钟。

• Routing Information Protocol (RIP) uses a periodic update to inform neighbors about a change in reachability (RIP doesn’t really carry topology information, though some information about the topology can be inferred from the database). The “default” timer for this periodic update is 30 seconds; on average (statistically), any given change in reachability will wait 15 seconds before being advertised to a neighboring router. The average time to notify the entire network of a change in reachability, then, is the maximum number of hops the reachability change must travel multiplied by 15 seconds. For large networks, this could mean two or three minutes.

• OSPF 是一种链路状态协议，因此拓扑和/或可达性的任何变化都会传播到整个网络。每个接收到更新的 LSA 的路由器将独立计算 SPT（使用 SPF 计算），这意味着（理论上）与洪泛域中的所有其他路由器并行。原来影响OSPF收敛的定时器是LSA生成定时器和SPF计算定时器。这两个计时器相结合，可以使标准的、未改变的 OSPF 的最短收敛时间约为一秒。此处显示的最大收敛时间约为 30 秒，这是在整个网络中发现和通告邻居故障所需的最长时间。

• OSPF is a link state protocol, so any change in topology and/or reachability is flooded through the entire network. Every router that receives an updated LSA will compute the SPT (calculated using SPF) independently, which means (theoretically) in parallel with all the other routers in the flooding domain. The original timers that impacted OSPF convergence are the LSA generation timer and the SPF calculation timer. These two timers, combined, result in a minimum convergence time for standard, unaltered OSPF of around one second. The maximum convergence time is shown here to be around 30 seconds, the maximum amount of time a neighbor failure will take to be discovered and advertised throughout the network.

• EIGRP 是一种距离矢量协议，它在本地数据库中保存一跳拓扑信息，并使用该信息在可能的情况下预先计算无环路路径。如果有备用无环路路径可用，则 EIGRP 可以在检测到链路故障后不到 100 毫秒 (ms) 内收敛。在没有 LFA（称为可行后继者，或 FS）的情况下，EIGRP 将通过网络传播查询；平均而言，此查询每跳需要大约 200 毫秒来处理。那么，EIGRP 收敛所需的平均时间为 200 毫秒乘以必须参与查询过程的路由器数量——由网络设计和配置调节的数量。最大收敛时间由 EIGRP Stuck in Active 定时器设置，通常为 90 秒左右。

• EIGRP is a distance vector protocol that holds one hop of topology information in a local database, using this information to precalculate loop-free paths where possible. If an alternate loop-free path is available, EIGRP can converge in less than 100 milliseconds (ms) after a link failure has been detected. In the absence of a LFA (called a Feasible Successor, or FS), EIGRP will propagate a query through the network; on average, this query will require about 200 ms per hop to process. The average amount of time EIGRP requires to converge is, then, 200 ms multiplied by the number of routers that must participate in the query process—a number regulated by network design and configuration. The maximum convergence time is set by the EIGRP Stuck in Active timer, which is normally around 90 seconds.

笔记

Note

为什么SPT计算理论上是并行的？由于一台路由器向其每个邻居洪泛数据包的速率以及通过多跳网络洪泛新信息所需的累积时间略有差异，因此无法确保网络中的所有路由器都收到新的拓扑信息恰好在同一时刻。鉴于此，也无法确保网络中的每个路由器都在同一时刻开始计算新的 SPT。这最终是链路状态协议收敛期间微循环的原因。

Why is SPT calculation theoretically in parallel? Because of slight differences in the rate at which one router can flood packets to each of its neighbors, and the cumulative time required to flood new information through a multihop network, there is no way to ensure all the routers in the network receive new topology information at precisely the same moment. Given this, there is also no way to ensure that every router in the network will begin calculating a new SPT at precisely the same moment. This is, ultimately, the cause of microloops during link state protocol convergence.

从图 5.7所示的应用程序需求中可以看出，这些收敛时间无法满足当今网络的预期。如何改善这些协议的收敛时间？这个过程的第一步是从说话转向更快说话。

As you can see from the application requirements shown in Figure 5.7, these convergence times just won’t supply what is expected out of networks today. How can convergence times for these protocols be improved? The first step in the process was to move from talking to talking faster.

通过智能定时器提高融合：说话速度更快

Improving Convergence with Intelligent Timers: Talk Faster

如果我们想提高路由协议的收敛性，最容易从哪里开始？鉴于重点关注各个计时器之间的交互、路由信息的分发以及新转发表的计算，最明显的起点是协议用来确定何时通告新信息的计时器。为了更好地理解缩短这些计时器时所面临的问题，我们首先需要了解计时器为何存在。让我们考虑在图 5.8所示的网络上运行的 OSPF 。

If we want to improve the convergence of a routing protocol, where would be the easiest place to start? Given the focus on the interaction between the various timers, the distribution of routing information, and the calculation of new forwarding tables, the most obvious place to begin would be with the timers the protocols use to determine when to advertise new pieces of information. To better understand the problem, we face when shortening these timers, we need to understand why the timers are there in the first place. Let’s consider OSPF running on the network shown in Figure 5.8.

图 5.8 简单的 OSPF 网络

Figure 5.8 A Simple OSPF Network

为了说明这一点，假设 2001:db8:0:1::/64 链路正在抖动，因此失败，然后每隔 200 或 300 毫秒重新连接一次。如果每次该链路状态发生变化时，Router A 都会生成一个新的 LSA，会发生什么情况？路由器 B 和 C 将被大量路由更新淹没，从而占用带宽和缓冲区空间。为了防止这种情况发生，OSPF 设计了路由器在通告链路状态更改之前等待的最短时间（LSA生成计时器）。如果 LSA 生成计时器设置得相当高，本地连接链路中的快速状态变化将被抑制——像这里描述的那样的持续抖动不会导致LSA 不断涌入网络。另一方面，将LSA生成定时器设置得太高会导致网络收敛速度变慢。我们如何解决这个问题呢？

Just to illustrate the point, assume the 2001:db8:0:1::/64 link is flapping so it fails, then reconnects, every 200 or 300 ms. What happens if Router A generates a new LSA each time this link changes state? Routers B and C would be flooded with routing updates, taking up bandwidth and buffer space. To prevent this from happening, OSPF is designed with a minimal amount of time a router will wait before advertising a link state change—the LSA generation timer. If the LSA generation timer is set fairly high, rapid status changes in locally connected links will be damped—constant flaps like the one described here will not cause a constant flood of LSAs flooded through the network. On the other hand, setting the LSA generation timer too high will cause the network to converge more slowly. How do we resolve this problem?

解决方案是使用可变计时器，允许路由器 A 在 2001:db8:0:1::/64 链路更改状态时非常快速（甚至立即）通告新的链路状态，但随后“退出”，因此路由器 A 必须等待一段较长的时间才能通告下一个链路状态更改。这将使网络能够针对单个更改快速收敛，但会减弱短时间内发生的大量更改的影响。

The solution is to use a variable timer—allow Router A to advertise the new link state very quickly (or even immediately) when the 2001:db8:0:1::/64 link changes state, but then “back off,” so that Router A must wait some longer period of time before advertising the next link state change. This will allow the network to converge quickly for single changes, but dampen the effect of a large number of changes happening over a short period of time.

这正是指数退避（目前大多数 OSPF 实现中包含的一项功能）所做的事情。计时器设置为一个非常低的数字，并且每次发生新事件时都会增加计时器的值，直到达到新广告之间的某个最大时间量。随着时间的推移，没有任何事件发生，计时器会减少，直到最终再次达到最小值。图 5.9说明了指数退避定时器的操作。

This is precisely what exponential backoff, a feature now included in most OSPF implementations, does. The timer is set to a very low number, and increased each time a new event occurs, until it reaches some maximum amount of time between new advertisements. As time passes with no events occurring, the timer is reduced until it eventually reaches the minimum again. Figure 5.9 illustrates the exponential backoff timer operation.

图 5.9 指数退避定时器操作

Figure 5.9 Exponential Backoff Timer Operation

在图 5.9中，2001:db8:0:1::/64 的第一次故障导致路由器 A 立即生成 LSA（接近），以便尽快通知网络的其余部分。RouterA发送该LSA后，将自己的LSA生成定时器修改为定时器步长1。然后，路由器 A 立即开始“衰减”LSA 生成计时器，随着时间的推移慢慢减少它，而不会发生任何新的链路状态变化。然而，过了一会儿，第二次拓扑发生变化。路由器 A 现在等待，直到计时器到期，发送新的 LSA，并为计时器添加更多时间，将其增加到计时器步骤 2图中。同样，路由器 A 开始慢慢减少计时器，同时 2001:db8:0:1::/64 链路保持稳定。然而，第三次状态更改会导致路由器 A 等待，直到计时器到期，传输新的 LSA，并再次为计时器值添加更多时间。此时，已达到最大计时器设置- 无论未来发生多少次故障，计时器都不会设置得高于此值。

In Figure 5.9, the first failure of 2001:db8:0:1::/64 causes Router A to generate an LSA (close to) immediately, so the rest of the network is informed as quickly as possible. Router A, after sending this LSA, modifies its LSA generation timer to Timer Step 1. Router A then immediately begins “decaying” the LSA generation timer, reducing it slowly as time passes without any new link state changes. After a few moments, however, the second topology change occurs. Router A now waits until the timer expires, sends out a new LSA, and adds more time to the timer, increasing it to Timer Step 2 in the illustration. Again, Router A begins reducing the timer slowly while the 2001:db8:0:1::/64 link remains stable. A third state change, however, causes Router A to wait until the timer expires, transmit a new LSA, and, again, add more time to the timer value. At this point, the maximum timer setting has been reached—no matter how many future failures occur, the timer will never be set any higher than this value.

这种类型的指数退避也可以应用于 SPT 计算之间的间隔，以便为少量变化提供快速反应，而不会因更新的信息和处理要求而使网络过载。

This type of exponential backoff can be applied to the interval between SPT calculations, as well, to provide fast reaction for small numbers of changes without overloading the network with updated information and processing requirements.

笔记

Note

如果这看起来类似于 BGP 抑制图表，那是因为抑制和指数退避使用相同的原理和技术来提高网络稳定性，同时允许快速通知可达性或拓扑的微小变化。

If this looks similar to a chart for BGP dampening, that’s because it is—dampening and exponential backoff use the same principles and techniques to promote network stability while allowing for fast notification of small changes in reachability or topology.

在说话速度更快和复杂性之间的关系中，有几点值得考虑。

Several points are worth considering in the relationship between talking faster and complexity.

• 仅向控制平面添加最少量的状态 — 事实上，根本没有向路由器之间传送的控制平面状态添加任何新状态。必须将许多额外的计时器添加到控制平面协议实现中（当然，这可能非常复杂，具体取决于计时器必须运行的粒度 - 每个邻居/对等点、每个前缀等），但这对于“在线”协议本身来说都是不透明的。线路上可能有（也可能没有）更多流量来提供更快的收敛，具体取决于计时器的设置方式。在较旧的网络中，传输速度较低，这些额外的数据包可能会对网络本身的运行产生很大影响。在更现代、更高速的链路中，相对于收敛速度的增益，这种额外的流量是最小的。这些类型的计时器修改，控制平面中的操作速度控制平面在整个网络中传播状态。随着事件发生率的增加（指数退避）而增加广告事件的间隔，可以降低操作速度，以至于如果将两种技术结合起来，几乎不会增加任何额外的复杂性。

• Only a minimal amount of state is added to the control plane—in fact, no new state is added to the control plane state carried between routers at all. A number of additional timers must be added to the control plane protocol implementation (this can be quite complex, of course, depending on the granularity at which the timers must operate—per neighbor/peer, per prefix, etc.), but this is all opaque to the “on the wire” protocol itself. There may—or may not—be more traffic on the wire to provide for faster convergence, depending on how the timers are set. In older networks, with lower transmissions speeds, these additional packets could have a large impact on the operation of the network itself. In more modern, higher speed links, this additional traffic is minimal against the gains in convergence speed. These sorts of timer modifications, done correctly, might increase the speed of operations in the control plane, the speed at which the control plane is propagating state throughout the network. Increasing the spacing of advertising events as the rate of events increases (exponential backoff), mitigates the speed of operations to the point that little additional complexity is added if the two techniques are combined.

• 也许最复杂的是通过控制平面中交互表面的指数退避计时器来增加的。运营商现在拥有大量设备，每个设备都具有相当独立的计时器，并且可以以不同的速率运行，而不是拥有大量具有相当一致的计时器的设备。这使得在任何给定时刻准确预测网络状态变得更加困难，在故障排除或确定特定情况下控制平面行为时，会引入更多的“量子状态”性质。与单一故障情况相比，多个故障情况现在将产生不同的事件链，可能导致难以跟踪竞争条件和复杂表面相互作用的其他工件。

• Perhaps the most complexity is added through exponential backoff timers along the interaction surfaces in the control plane. Rather than having a lot of devices with fairly consistent timers, the operator now has a lot of devices, each with fairly independent timers that could be running at different rates. This makes it harder to predict, at any given moment, precisely what the state of the network is, introducing a more “quantum state” nature to the mix when troubleshooting or determining what control plane behavior will be in specific situations. Multiple failure situations will now produce a different chain of events than single failure situations, potentially causing difficult to trace race conditions and other artifacts of complex surface interactions.

笔记

Note

与所有控制平面修改一样，在将指数退避配置为提供更快收敛的机制时，需要考虑明确的权衡。例如，单个行为不当的设备（或攻击者）可以通过将拓扑的错误更改注入到控制平面中，强制指数退避计时器保持在高位（接近图 5.9 中的最大计时器设置）。如果这些计时器被迫保持高于应有的水平，网络对实际变化的反应就会太慢，可能会导致应用程序故障。这似乎有点牵强，但它也不是典型的网络管理员在解决应用程序性能问题时会寻找的东西。

As with all control plane modifications, there is a definite tradeoff to be considered when configuring exponential backoff as a mechanism to provide faster convergence. For instance, a single misbehaving device (or an attacker) can, by injecting false changes to the topology into the control plane, conceivably, force the exponential backoff timers to remain high (near the maximum timer setting in Figure 5.9). If these timers are forced to remain higher than they should be, the network will react too slowly to real changes, potentially causing application failures. This might seem to be far-fetched, but it’s also not something the typical network administrator is going to look for when troubleshooting application performance issues.

因此，修改路由协议中的收敛计时器通常会在收敛时间方面提供大量增益，而不会产生巨大的复杂性影响。

Modifying the convergence timers in routing protocols, then, typically provides a lot of gain in terms of convergence time without having a huge complexity impact.

从收敛中删除计时器：预计算

Removing Timers from Convergence: Precompute

说得越来越快之后，接下来该做什么？如果使定时器更加智能可以显着缩短收敛时间，那么简单地使定时器完全脱离收敛又如何呢？这就是预先计算的 LFA 路径的作用。图 5.10将用于说明其工作原理。

What’s next after talking and talking faster? If making the timers more intelligent can dramatically improve convergence time, then what about simply taking the timers out of convergence altogether? This is what precomputed LFA paths does. Figure 5.10 will be used to illustrate how this works.

图 5.10 快速收敛的预计算示例

Figure 5.10 Precompute Example for Fast Convergence

在该网络中，Router A 有两条路径到达 Router C 后面的目的地：通过 Router B，总成本为 2；通过 Router D，总成本为 3。在这种情况下，Router A 显然会选择经过 Router C 的路径。 B 的总开销为2。但是为什么RouterA 不能将经过RouterD 的路由设置为备份路径呢？很明显，该路径是无环路的。

In this network, Router A has two paths to reach destinations behind Router C: via Router B with a total cost of 2, and via Router D with a total cost of 3. In this case, Router A will obviously choose the path through Router B with a total cost of 2. But why couldn’t Router A install the route through Router D as a backup path? It’s obvious the path is loop free.

事实上，这就是预计算的作用——发现备用的无环路路径并将它们安装为本地可用的备份路径。要记住的关键点是，路由器无法像网络工程师那样“看到”网络，通过网络端到端的实际图表来绘制到达单个目的地的备用路径（事实并非如此）无论如何，就像在地图上找到替代路径一样简单 - 路径的成本会产生很大的差异，正如我们将在下一节中看到的那样）。路由器所能做的就是检查两条路径的成本，尤其是任何备用路由的下一跳通告的成本，以确定该路径是否实际上应用作 LFA。

This is, in fact, what precomputing does—discover alternate loop-free paths and install them as a locally available backup path. The key point to remember is that routers can’t “see” the network like a network engineer, with an actual diagram of the network end-to-end on which to map out alternate paths to reach a single destination (and it’s not so simple as to find the alternate paths on a map, anyway—the cost of the path makes a big difference, as we’ll see in the next section). All the router can do is examine the cost of the two paths, especially the cost being advertised by the next hop of any alternate route, to determine if the path should actually be used as a LFA.

此时，Router A 可以查看Router D 的Cost，判断经过Router D 的路径不可能是环路，因为Router D 的路径Cost 小于Router A 的路径Cost。在 EIGRP 术语中，这称为可行性测试；如果邻居的报告距离（路由器 D 处的成本）小于本地可行距离（路由器 A 处的最佳路径），则该路径是无环路，可以用作备份。OSPF 和 IS-IS 计算 LFA 的方式大致相同，都是通过从邻居的角度计算到任何给定目的地的成本来确定路径是否无环路。

In this case, Router A can look at Router D’s cost and determine that the path through Router D cannot be a loop, because the cost of Router D’s path is less than the cost of Router A’s path. In EIGRP terms, this is called the feasibility test; if the neighbor’s Reported Distance (the cost at Router D) is less than the local Feasible Distance (the best path at Router A), then the path is loop free, and can be used as a backup. OSPF and IS–IS calculate LFAs in much the same way, by calculating the cost to any given destination from the neighbor’s point of view to decide if the path is loop free or not.

让我们从复杂性角度考虑 LFA 路径的预计算。

Let’s consider the precomputation of LFA paths in complexity terms.

• 与定时器修改一样，预计算LFA 路径不会向控制平面中通过网络承载的状态添加任何内容。计算这些备用路径所需的所有信息都可以在信息中找到已经由控制平面携带，包括 EIGRP 和链路状态协议，因此没有理由添加任何新信息。当然，预计算备用路径确实会增加任何特定实现的内部状态，从而引入需要测试的新代码路径。

• Like timer modification, precomputing LFA paths doesn’t add anything to the state carried through the network in the control plane. All the information required to compute these alternate paths is available in information already carried by the control plane, both with EIGRP and with link state protocols, so there is no reason to add any new information. Precomputing alternate paths does, of course, increase the internal state for any specific implementation, introducing new code paths that need to be tested.

• 预计算对控制平面的运行速度几乎没有影响——如果有的话，如果可以通过预计算保护通过网络的大多数路径，则上一节中讨论的定时器修改可能会变得不那么重要，从而使收敛速度变得更快“更懒”，控制平面中的通知速度更慢。

• Precomputation has little to no impact on the speed at which the control plane operates—if anything, if most paths through a network can be protected through precomputation, the timer modifications discussed in the previous section may become less important, allowing convergence speeds to become “lazier,” and the pace of notifications in the control plane to be slower.

• 也许最复杂的部分是通过沿着交互表面的预计算增加的——就像指数退避计时器一样在控制平面中。这在两个维度上都是如此——控制平面操作和操作状态。单个或多个故障可能导致重叠的预计算路径发挥作用，从而导致网络出现意外状态，从而使控制平面操作变得更加复杂。种族和其他条件是必须考虑（并可能进行测试）的可能副作用。从运行状态的角度来看，预先计算的路径为每个转发设备操作人员添加了多一位内部状态——什么是备用路径，网络是否处于切换到备用路径和计算新的最佳路径之间的状态，例如。如果两个故障同时或接近同时发生，操作员很难预测结果，

• Perhaps the most complexity is added through precomputation—as was true with exponential backoff timers—along the interaction surfaces in the control plane. This is true along two dimensions—control plane operation and operational state. Control plane operation is made more complex by the possibility that a single or multiple failures may cause overlapping precomputed paths to come into play, causing unanticipated states in the network. Race and other conditions are a possible side effect that must be considered (and possibly tested for). From an operational state perspective, precomputed paths add one more bit of internal state to each forwarding device operators must pay attention to—what is the alternate path, is the network in a state between switching to the alternate path and calculating a new best path, for instance. In the case of two failures occurring at or near the same time, it’s difficult for an operator to predict the outcome, and hence to know how the network will actually converge (will it converge within the required bounds).

与修改计时器以改进协议收敛的情况一样，预计算路径在明显复杂性的微小增加的情况下提供了大量收益。事实上，EIGRP 可行后继（一种预先计算路径的形式）已在许多大型网络中部署多年，复杂性几乎没有明显增加。

As was the case in modifying timers to improve protocol convergence, precomputing paths offers a lot of gains against very small increases in apparent complexity. In fact, EIGRP Feasible Successors, a form of precomputed paths, have been deployed in many large-scale networks for many years with little noticeable increases in complexity.

解决拓扑问题：通过隧道连接至无环路备用网络

Working around Topology: Tunneling to the Loop-Free Alternate

如果图 5.10中的指标稍微改变，添加一个额外的路由器，并产生如图 5.11所示的网络，会发生什么？

What happens if the metrics in Figure 5.10 are changed slightly, throwing in an extra router, and resulting in the network illustrated in Figure 5.11?

图 5.11 并非无循环的替代方案

Figure 5.11 The Loop-Free Alternate That Isn’t

Router A 仍然有两条路径到达 Router C 之外的每个目的地，例如 2001:db8:0:1::/64，但经过 Router D 的路径不能用作 LFA。为什么？如果经过RouterB到达2001:db8:0:1::/64的路径出现故障，RouterA切换到通过路由器 D 的备用路径，路由器 D 会将该流量转发到哪里？RouterD有两条路径到达2001:db8:0:1::/64；通过路由器 E，成本为 4，通过路由器 A，成本为 3。然后，路由器 D 会将路由器 A 发送的任何流量（目的地为 2001:db8:0:1::/64）转发回路由器A。

Router A still has two paths to every destination beyond Router C, such as 2001:db8:0:1::/64, but the path via Router D cannot be used as a LFA. Why? If the path to 2001:db8:0:1::/64 through Router B fails, and Router A switches to the alternate path through Router D, where will Router D forward this traffic? Router D has two paths to 2001:db8:0:1::/64; through Router E with a cost of 4, and through Router A with a cost of 3. Router D will, then, forward any traffic Router A sends it with a destination of 2001:db8:0:1::/64 back to Router A.

在这种情况下，路由器 A 使用路由器 D 作为 LFA，实际上会在经过路由器 B 的路径发生故障和路由器 D 重新计算其最佳路径并开始使用经过路由器 E 的路径之间的时间内导致路由环路达到2001:db8:0:1::/64。这称为微循环，因为循环是由控制平面引起的，并且持续时间往往很短（它不是永久循环）。

Router A using Router D as a LFA, in this case, actually results in a routing loop during the time between the failure of the path through Router B, and the time when Router D recomputes its best path and starts using the path through Router E to reach2001:db8:0:1::/64. This is called a microloop, because the loop is caused by the control plane, and it tends to be for a very short duration (it isn’t a permanent loop).

我们该如何解决这个问题？最明显的答案是不使用通过路由器 D 的路径作为备份路径，但我们试图在网络拓扑发生变化时提供快速的流量重新路由，而不是丢弃数据包。快速重路由解决方案可以通过隧道经过路由器 D 到达网络中的某个点来解决此问题，在该点，流量不会转发（循环）回路由器 A。有多种方法可以实现此目的：

How can we resolve this? The most obvious answer is to simply not use the path through Router D as a backup path—but we’re trying to provide for fast rerouting of traffic in the case of a change in the network’s topology, not dropped packets. Fast reroute solutions can solve this problem by tunneling past Router D, to some point in the network where the traffic won’t be forwarded (looped) back toward Router A. There are several ways to accomplish this:

• 从更广泛的邻居环的角度计算SPT，直到您找到几跳之外的一个点，您可以通过隧道将流量传送到该点，而不会使其循环回到您自己。大多数执行此操作的机制都会停止在两跳（您的邻居的邻居）处进行搜索，因为这会以这种方式找到可用的所有备用路径中的绝大多数。

• Compute the SPT from the perspective of a wider ring of neighbors until you find a point several hops away where you can tunnel traffic to without it looping back to yourself. Most mechanisms that do this stop searching at two hops (your neighbor’s neighbor), because this finds the vast majority of all alternate paths available in this way.

• 通过路由器 D 和 E 将 2001:db8:0:1::/64 的可达性通告给路由器 A。¹

• Advertise reachability to 2001:db8:0:1::/64 through Routers D and E to Router A.¹

1 . 例如，请参阅 S. Bryant、S. Previdi 和 M. Shand，“使用非通过地址的 IP 和 MPLS 快速重新路由框架”（IETF，2013 年 8 月），2015 年 9 月 15 日访问，https: // www.rfc-editor.org/rfc/rfc6981.txt。

1. See, as an example S. Bryant, S. Previdi, and M. Shand, “A Framework for IP and MPLS Fast Reroute Using Not-Via Addresses” (IETF, August 2013), accessed September 15, 2015, https://www.rfc-editor.org/rfc/rfc6981.txt.

• 使用诸如随机游走的深度优先搜索等算法来计算替代拓扑。²

• Compute an alternate topology using an algorithm such as depth first searching with a random walk, or others.²

2 . 例如，参见 A. Atlas 等人，“An Architecture for IP/LDP Fast-Reroute using Maximally Redundant Trees”（IETF，2015 年 7 月），2015 年 9 月 15 日访问，https: //www.ietf.org/ id/draft-ietf-rtgwg-mrt-frr-architecture-06.txt。

2. See, as an example A. Atlas et al., “An Architecture for IP/LDP Fast-Reroute Using Maximally Redundant Trees” (IETF, July 2015), accessed September 15, 2015, https://www.ietf.org/id/draft-ietf-rtgwg-mrt-frr-architecture-06.txt.

• 计算从目的地到本地路由器的反向树；在此反向树上找到不在从本地路由器到目的地的当前最佳路径上的点。³

• Compute a reverse tree from the destination toward the local router; find a point on this reverse tree that is not on the current best path from the local router toward the destination.³

3 . 有关更多信息，请参阅《网络架构的艺术：业务驱动设计》第一版中的第 8 章“风雨如磐” 。（印第安纳州印第安纳波利斯：思科出版社，2014 年）。

3. For further information, see Chapter 8, “Weathering Storms,” in The Art of Network Architecture: Business-Driven Design, 1st edition. (Indianapolis, Indiana: Cisco Press, 2014).

一旦找到可以安全传输流量的远程点，就必须自动创建某种形式的隧道以用作备份路径。该隧道必须以非常高的度量插入到本地转发表中，或者以防止在从表中删除主（非隧道）路径集之前使用它的方式插入。隧道的端点还必须有一个隧道尾端，通过它来处理沿着该路径转发的数据包，最好是自动配置的东西，以减少网络管理和配置开销。

Once the remote point where traffic can be safely tunneled is found, some form of tunnel must be automatically created to use as a backup path. This tunnel must be inserted in the local forwarding table with a very high metric, or inserted in such a way as to prevent it from being used until the set of primary (nontunneled) paths have been removed from the table. The end point of the tunnel must also have a tunnel tail end though which to handle packets being forwarded down this path, preferably something that is autoconfigured to reduce network management and configuration overhead.

回到到目前为止开发的状态、速度、表面、优化模型将有助于揭示这些类型的隧道快速重路由给网络带来的新复杂性。

Returning to the state, speed, surface, optimization model developed up to this point will help expose what new complexities these types of tunneled fast reroute add to the network.

• 隧道解决方案以多种方式添加状态。首先，存在额外的内部和实现状态，这两者都增加了复杂性。此状态包括当网络拓扑发生变化时重新计算任何隧道备份路径所需的额外处理。其次，一些机制需要显式地通告额外的控制平面状态（例如NotVia），这增加了控制平面协议的复杂性。第三，如果用于构建快速重路由备份路径的隧道协议当前未部署在网络中，则必须部署它们 - 向堆栈添加其他协议、配置等。最后，整个网络中的配置（或自动配置）隧道端点通过增加配置而增加了复杂性。

• Tunneled solutions add state in a number of ways. First, there is the additional internal and implementation state, both of which add complexity. This state includes additional processing required to recalculate any tunneled backup paths when the network topology changes. Second, some mechanisms require the advertisement of additional control plane state explicitly (such as NotVia), which adds control plane protocol complexity. Third, if the tunneling protocol used to build the fast reroute backup paths are not currently deployed in the network, they must be deployed—adding additional protocols to the stack, configurations, etc. Finally, the provisioning (or autoprovisioning) tunnel endpoints throughout the network increases complexity by increasing configuration.

• 隧道快速重路由机制不会提高控制平面的运行速度，也不会提高网络拓扑发生变化的速度。事实上，快速重路由实际上可以降低速度通过减少对上述花哨和调整的计时器的需求来进行更新。在这种情况下，隧道快速重路由机制可以是降低复杂性的一种方法。

• Tunneled fast reroute mechanisms don’t increase the speed at which the control plane operates, nor the speed at which changes occur in the network topology. In fact, fast reroute can actually reduce the speed of updates by reducing the need for the fancy and tuned timers described above. In this case, tunneled fast reroute mechanisms can be a way to reduce complexity.

• 隧道解决方案以多种方式增加交互表面的尺寸和深度。事实上，这可能是这些解决方案增加网络复杂性的主要方式。几个例子就足够了——首先，确定网络中的流量模式要复杂得多，特别是在网络中断期间。对于重视高水平利用率（而不是过度建设和忽视服务质量）的网络，这增加了在规划、配置和管理服务质量问题时需要考虑的许多因素。第二，因为流量可以在瞬态期间通过隧道传输到网络中多跳之外的点，因此跟踪特定流或流的流动可能会变得更加困难。在拓扑结构不断变化的网络中，这可能成为一项巨大的挑战。第三，在确定流量流向何处及其原因时，操作员必须记住查看多个位置 - 控制平面的内容不仅仅是当前转发表中的内容。最后，隧道机制要求大量设备能够动态终止隧道以重新路由流量。这些动态开放的隧道尾端中的每一个都是潜在的安全威胁和额外的管理复杂性。

• Tunneled solutions increase the size and depth of interaction surfaces in a number of ways. In fact, this is probably the primary way in which these solutions increase network complexity. A few examples will suffice—First, determining the traffic patterns in a network is much more complex, particularly during network outages. For networks that value a high level of utilization (rather than overbuilding and ignoring quality of service), this adds a number of factors to consider when planning, configuring, and managing quality of service issues. Second, because traffic can travel over tunnels during transient states to a point multiple hops away in the network, following the flow of a particular stream or flow can become much more difficult. In networks with constant topology change, this can become an overwhelming challenge. Third, operators must remember to look in more than one place when determining where traffic is flowing and why—there is more to the control plane than just what is in the current forwarding table. Finally, tunneled mechanisms require that a large number of devices be capable of dynamically terminating tunnels to reroute traffic. Each of these dynamically open tunnel tail ends are potential security threats and additional management complexity.

关于快速收敛的一些最终想法

Some Final Thoughts on Fast Convergence

现在应该很明显，从默认计时器到预先计算的 LFA，再到隧道 LFA，涉及到复杂性上的两个稍微小的步骤，以及复杂性稍大的最后一步。图 5.12说明了这个概念。

It should be obvious by now that moving from default timers, to precalculated LFAs, to tunneled LFAs, involves two somewhat small steps up in complexity, and one final step that’s a bit larger in complexity. Figure 5.12 illustrates this concept.

图 5.12 复杂性与收敛性

Figure 5.12 Complexity versus Convergence

笔记

Note

隧道 LFA 和预计算 LFA 在每个解决方案提供的收敛速度上几乎相同，因此它们在图 5.12中紧密地显示在一起。这两者之间的主要权衡是控制平面的复杂性与各自将覆盖的拓扑类型。隧道式 LFA 可以覆盖几乎所有拓扑（有一些例外，非平面拓扑并不总是被所有隧道式 LFA 技术覆盖），但代价是控制平面更加复杂。

Tunneled and precomputed LFAs are close to identical in the speed of convergence offered by each solution, so they are shown close together in Figure 5.12. The primary tradeoff between these two is the amount of control plane complexity versus the types of topologies each will cover. Tunneled LFAs can cover just about every topology (with some exceptions—nonplanar topologies aren’t always covered by every tunneled LFA technology), at the cost of more complexity in the control plane.

您可以将每个特定的解决方案放置在该曲线上的不同位置 - 这完全取决于您正在使用的网络的要求，以及您如何看待向更快的融合所增加的复杂性。无论哪种方式，沿着通话连续体、更快通话、预计算和隧道的移动都会将另一层互连实体作为系统、另一组交互表面和另一组不确定点添加到网络中。虽然每一个表面上看起来都相当简单，但当试图快速收敛时，复杂性会迅速增加。这里的权衡是真实的、持久的和紧迫的。

You might place each specific solution in a different place on this curve—it all depends on the requirements of the network you’re working on, and how you perceive the added complexity of moving to ever faster convergence. Either way, movement along the continuum of talk, talk faster, precompute, and tunnel adds another layer of interconnected entities into the network as a system, another set of interaction surfaces, and another group of uncertainty points. While each one appears to be fairly simple on the surface, complexity can add up very quickly when trying to converge quickly. The tradeoffs here are real, persistent, and urgent.

快速融合的游戏会有终结吗？让我们反过来问这个问题。网络本质上是安抚人类的急躁情绪。网络工程师面临的问题是，人类的不耐烦没有明显的终点——无论事情有多快，它总是可以更快。正如本章开头所指出的，电子邮件和文件传输可以在控制平面收敛速度较快的情况下正常工作。然而，语音和视频对网络提出了更高的期望。股市的高速交易和实时医疗应用对网络融合提出了更高的要求。随着机器对机器和其他应用程序跨网络部署，它们可能会推动收敛速度不断提高，直到工程师们只是达到了使网络控制平面更快收敛的极限。简而言之，让网络更快收敛的努力可能没有尽头。

Will there ever be any end to the fast convergence game? Let’s put the question the other way around. Networks are, in essence, a sop to human impatience. The problem network engineers face is that there is no apparent end point in human impatience—no matter how fast something is, it can always be faster. As noted in the beginning of this chapter, email and file transfer work fine with moderately fast control plane convergence. Voice and video, however, put higher expectations on the network. High-speed trading in the stock market and real-time medical uses place even harder requirements on network convergence. As machine to machine and other applications are deployed across networks, they are likely to push ever increasing convergence speeds, until engineers have simply reached the limit of what can be done to make a network control plane converge faster. In short, there is probably no end in sight for trying to make the network converge more quickly.

虚拟化与设计复杂性

Virtualization versus Design Complexity

虚拟化是网络世界中“永远都是新技术”的技术之一。当帧中继首次提供虚拟电路时，它是新的；当异步传输模式（ATM）首次提出时，它又是新的；当以太网上的VLAN被提出时，它也是新的；当MPLS（最初被称为标签交换）被提出时，它也是新的。，并且在将来的某个实例中它将再次成为新的。为什么虚拟化是网络工程中如此流行且经久不衰的话题？因为它提供了一种隐藏信息的方法（更多关于信息隐藏的信息可以在第 6 章“管理设计复杂性”中找到）”），它允许网络运营商以看似简单的方式解决复杂的问题。为了理解虚拟化背后的原因，我们来看一个使用图 5.13所示网络的简单用例。

Virtualization is one of those “always new” technologies in the world of networking. It was new when Frame Relay first provisioned virtual circuits, it was new again when Asynchronous Transfer Mode (ATM) was first proposed, it was new when VLANs on Ethernet were proposed, and it was new when MPLS originally known as tag switching) was proposed, and it will be new again in some future instance. Why is virtualization such a popular and perennial topic in network engineering? Because it provides a way to hide information (more on information hiding can be found in Chapter 6, “Managing Design Complexity”), which allows network operators to resolve complex problems in apparently simple ways. To understand the reasoning behind virtualization, let’s look at a simple use case using the network illustrated in Figure 5.13.

图 5.13 虚拟化用例

Figure 5.13 Virtualization Use Case

假设我们想要确定主机 A 和 F 可以相互通信，主机 B 和 E 可以相互通信，但主机 A 和 B 不能，A 和 E 也不能，B 和 F 也不能，E 和 F 也不能。当然，我们可以在网络的所有接口上设置过滤器，以确保流量只能在我们决定应该通信的主机之间传递，而流量不能在我们不想通信的主机之间传递。然而，这将需要大量的管理和配置工作，创建两个虚拟拓扑（只有每个虚拟拓扑中的设备可以在其中进行通信）并将正确的主机集连接到每个虚拟拓扑会更简单。

Assume we want to be certain that Hosts A and F can talk to one another, and Hosts B and E can talk to one another, but Hosts A and B cannot, nor A and E, nor B and F, nor E and F. We could, of course, set up filters at all the interfaces through the network to ensure traffic can only pass between the hosts we’ve decided should communicate, and traffic cannot pass between hosts we don’t want to communicate. This would take a lot of management and configuration effort, however—it’s simpler to create two virtual topologies on which only devices within each virtual topology can communicate, and attach the right set of hosts to each virtual topology.

在这种情况下，我们可以在一个拓扑中包括 E（它是上游路由器）、D（[D,E] 链路）、B 的上游路由器、C（[B,C] 链路）和 B。在另一种拓扑上我们可以包括 A、A 的上游路由器、C、[A,C] 链路、D、F 的上游路由器、[D,F] 链路和 F。只要我们有某种方法沿着 [ C，D]链接，因此每个拓扑似乎都是它自己的“网络”，我们可以构建两个明显独立的拓扑来承载流量。

In this case, we could include E, it’s upstream router, D, the [D,E] link, B’s upstream router, C, the [B,C] link, and B, on one topology. On the other topology we could include A, A’s upstream router, C, the [A,C] link, D, F’s upstream router, [D,F] link, and F. So long as we have some way to separate the two topologies along the [C,D] link, so each topology appears to be its own “network,” we can build two apparently independent topologies through which to carry traffic.

关于使用虚拟拓扑实现安全性的进一步思考

Further Thoughts on Using Virtual Topologies for Security

即使从安全角度来看，前面的示例也简化了虚拟化网络的许多用例。现实世界中经常使用的一个示例是将公共可访问的服务器和服务（例如面向公众的 Web 服务器）与私有支持的服务器（例如数据库或业务逻辑服务）分开。通过将应用程序分解为多个组件，可以使用公共池来解决那些需要公开访问的部分，而那些不应该公开访问的部分可以从私有池中解决。将此最大化通常会导致微服务的创建，⁴非常小的服务，每个服务都有一个特定的目的，仅通过网络进行通信；此类服务的接口不仅需要防止公共访问，而且还需要防止内部用户访问。微分段将微服务的概念带入另一个领域，将单个网络分解为虚拟拓扑，并且仅当用户和应用程序需要驻留在其中的服务时才允许用户和应用程序访问这些拓扑。与文本中给出的示例非常相似，这提供了另一种安全途径，不仅可以防止未经授权的访问，还可以阻止任何内部或外部攻击者甚至访问包含关键服务的进程。

The preceding example provides simplistic reductions of the many uses cases for virtualized networks, even from a security point of view. One example that is used in the real world on a regular basis is separating publicly accessible servers and services (such as a public facing web server) from private backed servers (such as a database or business logic service). By breaking an application apart into multiple components, those parts which need to be publicly accessible can be addressed using public pools, and those parts that should not be publicly accessible can be addressed out of private pools. Taking this to maximum scale often results in the creation of microservices,⁴ very small services, each with a focused purpose, that only communicate over the network; the interfaces for such services need to be protected not only from public access, but also from internal users. Carrying the concept of microservices into another realm, microsegmentation breaks a single network up into virtual topologies, and gives users and applications access to these topologies only when they need the services that reside there. Much like the example given in the text, this provides an alternate route to security that not only protects against unauthorized access, but also blocks any internal or external attacker’s ability to even reach the processes that house critical services.

4 . 有关微服务的精彩介绍，请参阅 Sam Newman，《构建微服务》，第一版。（奥莱利媒体，2015 年）。

4. For an excellent introduction to microserves, see Sam Newman, Building Microservices, 1st edition. (O’Reilly Media, 2015).

此类设计还可以通过节省地址空间并将大型故障域分解为许多不同的故障域来实现扩展。

These sorts of design enable scaling, as well, by conserving address space, and breaking what could be a large failure domain up into potentially many different failure domains.

虽然这有点过于简单化，但这就是大多数虚拟化部署的本质——创建虚拟拓扑的能力使我们能够将配置大量过滤器的复杂性转移到一组额外的控制平面和数据平面原语的复杂性。

While this is a bit oversimplified, this is the essence of most virtualization deployments—the ability to create virtual topologies allows us to transfer the complexity of configuring a lot of filters into the complexity of an additional set of control plane and data plane primitives.

虽然进行虚拟化的原因有很多，但它们都归结为某种形式的多路复用，无论是在应用程序、用户组、用户还是其他一些分界线之间。虚拟化也有多种形式，但它们都归结为同一组概念——放置一个输出包装器、隧道标头、或数据包标头中的其他一些标记，以将一种拓扑与另一种拓扑分离。让我们从这个相当简单的角度来考虑虚拟化的一些权衡。

While there are many reasons to virtualize, they all come down to some form of multiplexing or another, whether between applications, groups of users, users, or some other dividing line. There are also many forms of virtualization, but they all come down to the same set of concepts—putting an out wrapper, tunnel header, or some other maker in the packet header to segregate one topology from another. Let’s consider some of the tradeoffs of virtualization from this rather simplistic viewpoint.

功能分离

Functional Separation

虚拟化允许网络运营商将单个问题分解为多个部分，然后分别解决每个部分。一个简单的例子 - 不是管理来自数十个不同应用程序的数百个不同流的服务质量和交互，而是可以围绕沿着一个虚拟拓扑运行的一小组应用程序进行配置，然后单独管理虚拟拓扑间的配置。这种功能分层本质上是我们在构建协议、分层网络设计，甚至基于单元而不是整体的软件构建和测试中看到的。功能分离是一个主要的简化工具；虚拟化是一种经过时间考验的提供功能分离的良好方法。

Virtualization allows the network operator to break a single problem into multiple parts, and then solve each one separately. An easy example—rather than managing the quality of service and interactions of hundreds of different flows from dozens of different applications, configurations can be molded around a small set of applications traveling along one virtual topology, and then manage inter-virtual topology configuration separately. This layering of functions is essentially what we see in building protocols, hierarchical network designs, and even the building and testing of software based on units, rather than as monoliths. Functional separation is a major simplification tool; virtualization is a good, time tested way to provide functional separation.

然而，将层和问题分成多个部分可能会增加必须相互作用的表面的数量，从而产生更多的复杂性。创建的每个组件还会在新模块和其他现有模块之间创建一组交互点。回到服务质量的例子，虽然需要管理的流量和应用程序较少（通过虚拟化将它们分组），但仍需要回答许多新问题，例如：

Separating layers and problems into multiple parts, however, can generate more complexity by increasing the number of surfaces which must interact. Each component created also creates a set of points of interaction between the new module and other existing modules. To return to the quality of service example, while there are fewer traffic flows and applications to manage—they’re grouped off by virtualizing them—a number of new questions need to be answered, such as:

• 每个虚拟拓扑是否应被视为单个“服务类别”，还是每个虚拟拓扑内应有多个“桶”或服务类别？

• Should each virtual topology be considered a single “class of service,” or should there be multiple “buckets” or classes of service within each virtual topology?

• 如果每个虚拟拓扑将具有多个服务类别，那么应如何指示这些服务类别？将服务质量信息从“内部”标头复制到“外部”标头是否足够？这两条服务质量信息之间是否需要某种形式的映射？

• If there each virtual topology is going to have multiple classes of service, how should these classes of service be indicated? Is copying the quality of service information from the “inner” header to the “outer” header enough? Does there need to be some form of mapping between these two pieces of quality of service information?

• 不同类别的服务应如何在虚拟拓扑之间交互？是否应该将所有“黄金”服务集中到底层传输网络上的一个类别中，还是应该一种拓扑优先于其他拓扑（以及为什么）？当每个拓扑代表不同的客户时，这尤其成问题。如果两个客户都购买了“黄金”服务级别，当链路或路径严重拥塞以致无法维持此服务级别时，网络应如何反应对于两个客户？一个客户是否应该“输给”另一个客户？两者都应该接受降级服务吗？根据客户的规模或总价值，“黄金”类别中是否应该有不同的级别？这些都是很难回答的问题。

• How should the different classes of service interact between the virtual topologies? Should all the “gold” service be lumped into a single class on the underlying transport network, or should one topology have priority over the others (and why)? This is specifically problematic when each topology represents a different customer. If two customers have both purchased a “gold” level of service, how should the network react when a link or path is congested enough that this level of service cannot be maintained for both customers? Should one customer “lose” to the other? Should both receive degraded service? Should there be different levels within the “gold” class based on the size or total worth of the customer? These are difficult questions to answer.

转发平面复杂度

Forwarding Plane Complexity

转发平面更简单，因为网络中的每个数据包转发设备只需检查一小组标头位、标签或标记，以确定数据包可以在哪些链路上转发或不可以在哪些链路上转发。当然，另一种选择是拥有一组相当明确的过滤器，基于每个主机构建，转发设备必须检查这些过滤器才能做出相同的决定。考虑到维护和搜索此类列表的复杂性，报头位或外部报头是流量分离问题的更简单的解决方案。

The forwarding plane is simpler because each packet forwarding device in the network must only examine a small set of header bits, a label, or a tag, to determine which links a packet can be forwarded on or not. The alternative, of course, is to have a rather explicit set of filters, built on a per host basis, which forwarding devices must examine to make the same determination. Given the complexities of maintaining and search such a list, header bits or an outer header is a much simpler solution to the traffic separation problem.

另一方面，转发平面在某些方面更为复杂；流量在被网络接受时，必须对每个数据包施加一些位、标签或外部标头。当数据包传出网络时，必须剥离这些相同的信息位。

On the other hand, the forwarding plane is more complex in some ways; traffic, on being accepted into the network, must have some set of bits, a label, or an outer header imposed on each packet. These same bits of information must be stripped as the packet passes out of the network.

控制平面复杂性

Control Plane Complexity

虚拟化通过将通常在多个不同控制平面之间的单个控制平面中承载的可达性、拓扑和策略信息进行分段来简化控制平面。因此，虚拟化堆栈中的每个单独的控制平面都更加简单，但是整体网络的复杂性又如何呢？添加更多相互堆叠的控制平面显然会增加复杂性，具体如下：

Virtualizing simplifies the control plane by segmenting off reachability, topology, and policy information normally carried in a single control plane among several different control planes. Each individual control plane in a virtualized stack is therefore simpler—but what of the overall network complexity? Adding more control planes stacked on top of each other clearly increases complexity by the following:

• 增加状态数量。影响多个拓扑的链路和节点状态必须在多个控制平面中承载。

• Increasing the amount of state. Link and node status impacting multiple topologies must be carried in multiple control planes.

• 加快变革速度。虽然发生变化的实际速率可能保持不变，但报告这些变化的速度（控制平面集中的信息流速度）必须增加。

• Increasing the speed of change. While the actual rate at which changes take place might remain the same, the speed at which these changes are reported (the velocity of information flow in the set of control planes) must increase.

• 增加整个网络的交互面。多个控制平面，每个控制平面都有自己的定时和收敛特性，现在必须在同一组拓扑信息上运行。wait for bgp等解决方案旨在管理这些互控平面交互，但这些解决方案也不可避免地增加了状态复杂性。

• Increasing the interaction surfaces throughout the network. Multiple control planes, each operating with their own timing and convergence characteristics, must now operate over the same set of topology information. Solutions such as wait for bgp are designed to manage these intercontrol plane interactions—but these solutions inevitably add state complexity, as well.

等待 BGP

Wait for BGP

等待 BGP是虚拟化形式中两个运行在彼此之上的控制平面之间交互的完美示例（BGP 用于承载全局可达性，而 IGP 用于承载本地可达性，这就是虚拟化的一种形式）。图5.14说明了IGP和BGP之间的交互，以及等待BGP提供的解决方案。

Wait for BGP is a perfect example of the interaction between two control planes running on top of one another in one form of virtualization (BGP is used to carry global reachability, while IGPs are used to carry local reachability, making this a form of virtualization). Figure 5.14 illustrates the interaction between the IGP and BGP, and the solution wait for BGP provides.

图5.14 等待BGP

Figure 5.14 Wait for BGP

在此示例中，我们使用 2001:db8:0:2::/64 和 2001:db8:0:1::/64 之间穿过 [A,B,C] 的最佳路径。请注意，在 iBGP 中，从路由器 A 的角度来看，2001:db8:0:1::/64 的下一跳是路由器 C 上面向云的接口，而不是云内或云外的某个位置，并且最终路由器位于云中。路径（直接连接到目的地）。另请注意，2001:db8:0:1::/64 不是底层 IGP 中可到达的目标；路由器 A、B 和 D 到达此目的地的唯一路径是通过 BGP。

In this example, we being with the best path between 2001:db8:0:2::/64 and 2001:db8:0:1::/64 passing through [A,B,C]. Note that in iBGP, from Router A’s perspective, the next hop for 2001:db8:0:1::/64 is the interface on Router C facing the cloud, rather than someplace within or beyond the cloud, and the final router in the path (which directly connects to the destination). Note also that 2001:db8:0:1::/64 is not a reachable destination in the underlying IGP; the only path Routers A, B, and D have to this destination is through BGP.

如果路由器 B 发生故障，则 IGP 将快速找到一条经过 D 到 C 的新路由。路由器 A 到下一跳的路径现在将是 [A,D,C]，而不是 [A,B,C]。由于B和D上的BGP进程通过BGP都可达2001:db8:0:1::/64，因此RouterA将路径切换到RouterD作为BGP下一跳，对流量转发没有影响。

If router B fails, then the IGP will quickly find a new route toward C through D. Router A’s path to the next hop will now be [A,D,C], rather than [A,B,C]. As the BGP process on both B and D have reachability to 2001:db8:0:1::/64 through BGP, Router A switching it’s path toward Router D, as the BGP next hop, has no effect on traffic forwarding.

然而，当路由器 B 重新投入使用时，出现了问题。IGP 将快速重新计算路由器 A 和 C 之间的路径，将这两个路由器之间的流量切换回通过路由器 B 的路径。iBGP 不会那么快收敛 — 路由器 B 重新学习通往 2001:db8 的路由需要一些时间： 0:1::/64 通过其与路由器 C 的 iBGP 对等会话。在路由器 A 使用经过路由器 B 的路径到达其下一跳（通往 2001:db8:0:1::/64）的时间段内，并且路由器 B 的 iBGP 表正在重新收敛，路由器 B 将接收它不知道如何转发的流量 - 因此，该流量将被丢弃。

When Router B comes back into service, however, there is a problem. The IGP will recalculate the path between Routers A and C quickly, switching traffic between these two routers back to the path through Router B. iBGP will not converge as quickly—it will take some time before Router B relearns the route toward 2001:db8:0:1::/64 through its iBGP peering session with Router C. During the period of time when Router A is using the path through Router B to reach its next hop toward 2001:db8:0:1::/64, and router B’s iBGP table is reconverging, Router B will be receiving traffic it does not know how to forward—hence, this traffic will be dropped.

为了解决这个问题，大多数主要路由器供应商都实现了一项功能，允许路由器 B 上的 IGP 等待向路由器 A 通告路由器 C 的可达性，直到本地 BGP 进程发出信号表示其已收敛 - 等待 BGP。此功能会减慢网络上运行的两个控制平面之一的速度，以允许第二个控制平面“赶上”——这是一个可靠的想法，可以防止运行中的 BGP 网络上出现大量数据包丢失。

To resolve this, most major router vendors implement a feature that allows the IGP on Router B to wait in advertising reachability to Router C toward Router A until the local BGP process signals it has converged—wait for BGP. This feature slows down one of the two control planes in operation on the network to allow the second control plane to “catch up”—a solid idea that prevents a lot of dropped packets across BGP networks in operation.

命运共同体风险群体

Shared Fate Risk Groups

虽然共享风险链路组 (SRLG) 可能是虚拟化最容易理解的结果，但它们在网络设计和实施中通常是最难记住和解释的。事实上，有时，除非共同风险因重大停电而暴露出来，否则它们是不可能被发现的。本质上，每当两个虚拟链路共享相同的物理基础设施时，就会形成 SRLG。例如：

While Shared Risk Link Groups (SRLGs) are probably the most easily understood result of virtualization, they are often the most difficult to remember and account for in network design and implementation. Sometimes, in fact, they are impossible to discover until the common risk has reared its ugly head through a major outage. Essentially, an SRLG is formed whenever two virtual links share the same physical infrastructure. For instance:

• 在同一帧中继链路/节点上运行的两个不同电路。

• Two different circuits running over the same Frame Relay link/node.

• 两个不同的VLAN 在同一物理以太网链路上或同一交换机上运行。

• Two different VLANs running over the same physical Ethernet link, or across the same switch.

• 两个不同的虚拟进程在同一物理计算和存储集上运行。

• Two different virtual processes running on the same physical compute and storage set.

通常，SFRG 被隐藏在虚拟化层之下，以至于很少有设计人员考虑它们的存在或影响。在某些情况下，SRLG 可能会隐藏在合同和分包合同中——两个主要提供商通过公共光缆将冗余电路出租给单个设施，或者两个托管服务从单个数据中心提供冗余服务。网络设计者需要找到这些类型的重叠服务产品——例如，很少有提供商能够帮助确定哪些设施是共享的，哪些不是。

Quite often, SFRGs are buried under layers of virtualization to the point that very few designers think about their existence or impact. In some cases, SRLGs can be buried under contracts and subcontracts—two major providers leasing redundant circuits to a single facility across a common fiber cable, or two hosting services providing redundant services out of a single data center. It’s up to the network designer to find these sorts of overlapping service offerings—very few providers are going to be helpful in determining which facilities are shared, and which are not, for instance.

最后的想法

A Final Thought

设计复杂性并不总是显而易见的，因此并不总是被完全考虑或处理。事实上，意外后果法则，即面对困难问题时无法解决或消除复杂性的副作用，使得许多设计问题变得最令人难以忘怀且难以解决。在下一章中，我们将介绍网络工程师拥有的工具，这些工具可以帮助平衡复杂性与其他因素。

Design complexity is not always obvious, and therefore not always completely considered or dealt with. In fact, the law of unintended consequences, a side effect of the impossibility of solving or removing complexity in the face of hard problems, makes many of these design problems the most haunting, and difficult to solve. In the next chapter, we’re going to look at the tools network engineers have that can help balance complexity against the other factors.

第 6 章管理设计复杂性

Chapter 6. Managing Design Complexity

上一章探讨了工程师和架构师在设计网络拓扑和控制平面时需要考虑的权衡。网络工程师如何管理这些权衡——状态与延伸、拓扑与收敛速度、虚拟化与设计复杂性？什么时候应该选择一种（例如虚拟化）而不是另一种（例如命运共享）？事实上，这些是网络工程领域中最困难的决策之一，甚至可能更困难，因为它们很少作为有意的决策浮出水面。

The last chapter examined the tradeoffs engineers and architects need to consider when designing the topology and control planes of a network. How can the network engineer manage these tradeoffs—state versus stretch, topology versus the speed of convergence, and virtualization versus design complexity? When should one (for instance, virtualization) be chosen over the other (for instance, fate sharing)? These are, in fact, some of the most difficult decisions to make in the field of network engineering, perhaps even more so because they rarely come to the surface as intentional decisions at all.

本章将考虑工程师管理此类决策的几种方法，从传统到非传统。考虑的三个具体领域如下：

This chapter will consider a few ways engineers can manage these types of decisions, from the traditional to the not-so-traditional. Three specific areas considered are as follows:

• 模块化，可实现信息隐藏。

• Modularity, which enables information hiding.

• 信息隐藏作为管理控制平面状态的潜在解决方案。

• Information hiding as a potential solution to managing control plane state.

• 模型和可视化工具作为解决虚拟化和命运共享问题的方法。

• Models and visualization tools as a method to work with virtualization and fate sharing issues.

模块化

Modularity

模块化是网络设计和工程中长期使用的、经过验证的、真实的方法。让我们考虑一些模块化设计的例子，然后考虑模块化设计如何解决复杂性问题。

Modularity is a longstanding, tried, and true method used in network design and engineering. Let’s consider some examples of modular design, and then consider how modular design attacks the complexity problem.

均匀度

Uniformity

一致性是网络设计人员和工程师用来降低网络复杂性的常用机制。以下各节讨论其中几种技术及其权衡。

Uniformity is a common mechanism used by network designers and engineers to reduce complexity within a network. The following sections discuss several of these techniques, and their tradeoffs.

制服供应商

公司或网络运营团队通常选择同一供应商的设备在整个网络中使用，以减少接口和实施的数量，从而减少配置新节点和故障排除所需的培训量和各种技能集出现问题时。

It’s common for a company or network operations team to choose devices from the same vendor for use throughout their network to reduce the number of interfaces and implementations, thus reducing the amount of training and variety of skill sets required to configure new nodes as well as troubleshoot problems as they arise.

选择此方法来控制复杂性时需要进行三个权衡：

There are three tradeoffs when choosing this route to controlling complexity:

•供应商锁定：如果整个网络选择单一供应商的设备，那么该供应商最终将驱动硬件和软件的生命周期。如果没有大量的自我控制，供应商最终还将控制网络架构，无论这是否对公司来说是最好的。

• Vendor Lock-in: If a single vendor’s equipment is chosen throughout the network, then the vendor ends up driving the hardware and software life cycles. Barring a great deal of self-control, the vendor will also end up controlling the architecture of the network, whether or not that’s best for the company.

•成本：使用单一供应商几乎总是会增加成本，因为该供应商不面临竞争。

• Cost: Using a single vendor will almost always drive up cost, as the vendor faces no competition.

•单一文化失败：单一供应商的设备运行单一操作系统和单一应用程序集，属于单一文化。单一文化会遭受所有设备共享的一组故障模式的困扰；如果网络中的一个设备对一组特定情况反应不佳，则网络中的所有设备都会反应不佳。在单一文化中，看似小问题可能会演变成重大失败。

• Monoculture Failures: A single vendor’s equipment, running a single operating system, and a single set of applications, is a monoculture. A monoculture suffers from a shared set of failure modes across all the devices; if one device in the network reacts poorly to a specific set of circumstances, all the devices in the network will. Seemingly small problems can turn into major failures in a monoculture.

统一硬件

在单个数据中心结构中使用相同的物理交换机是实现一致性的一种方法。基于 Charles Clos 在 1952 年提出的原则，脊叶设计最初是为了构建基于最小同等尺寸交换机的大型结构，如图 6.1所示。

Using the same physical switches throughout a single data center fabric is one approach to uniformity. Based on the principles laid out by Charles Clos in 1952, spine and leaf designs were originally created to build a large fabric based on minimal equally sized switches, as shown in Figure 6.1.

图 6.1 Clos 或脊叶拓扑

Figure 6.1 A Clos, or Spine and Leaf, Topology

假设图 6.1中所示的每个叶节点到某种负载的连接只有 4 个，则所示 Clos 结构中的每个交换机都具有相同数量的接口 — 8 个。使用单一设备类型（每个设备有 8 个接口），总共可以互连 32 个设备（内部或不连接到外部网络）。事实上，最初设计的全部目的是让通过结合小型、同等尺寸交换机的交换功能来互连大量设备。以这种方式在整个数据中心使用单一设备类型允许网络运营商最大限度地减少现有备件的数量，并最大限度地减少更换故障设备或链路硬件时涉及的配置和管理。

Assuming that there are only four connections from each leaf node shown in Figure 6.1 to some sort of load, each switch in the Clos fabric illustrated has the same number of interfaces—8. Using a single device type, each with 8 interfaces, a total of 32 devices can be interconnected (internally, or not connected to an outside network). In fact, the entire point of the original design was to allow the interconnection of a large number of devices by combining the switching capabilities of small, equally sized switches. Using a single device type throughout an entire data center in this way allows the network operator to minimize the number of spares on hand, as well as minimize the configuration and management involved when swapping out a failed device or link hardware.

笔记

Note

Clos 结构可能看起来类似于传统的分层以太网或 IP 网络设计，特别是当它的所有叶节点都位于一侧时（在“折叠”配置中）。然而，有几个关键的区别。例如，在主干和叶子拓扑（或结构）中，任何两个叶子节点或任何两个主干节点之间不存在交叉连接。此属性意味着除了通过叶节点的拓扑环路之外，结构中不存在拓扑环路。如果叶节点配置为无法将连接到主干节点的接口上收到的流量转发回另一个主干节点，则主干和叶节点设计没有拓扑环路。此属性简化了网络的设计，允许其通过添加更多主干和叶节点进行扩展，并减少控制平面上的工作。

The Clos fabric may appear similar to a traditional hierarchical Ethernet or IP network design, particularly when it is drawn with all the leaf nodes on one side (in the “folded” configuration). However, there are several crucial differences. For instance, in the spine and leaf topology (or fabric), there are no cross connections between any two leaf nodes, or any two spine nodes. This property means there are no topological loops in the fabric other than those passing through a leaf node. If the leaf nodes are configured so they cannot forward traffic received on an interface connected to a spine node back to another spine node, the spine and leaf design has no topological loops. This property simplifies the design of the network, allowing it to scale through the addition of more spine and leaf nodes, and reducing the work on the control plane.

一些现实使得降低复杂性的尝试比表面上看起来更加困难。例如，设备模型随着时间的推移而变化，这意味着需要认真考虑控制网络中安装的设备的生命周期。抛开理论不谈，在每个位置使用单个设备是非常困难的，即使是在数据中心结构中也是如此。主干交换机通常需要比架顶设备更多的端口数，胖树设计迫使设计人员在多个较便宜的固定配置设备之间进行选择，或者更一致地使用更昂贵的刀片或基于插槽的设备等。

Several realities make this attempt at reducing complexity more difficult than it might appear on the surface. For instance, device models change over time, which means serious thought needs to be put into controlling the life cycle of devices installed in the network. Theory aside, it’s very difficult to use a single device in every location, even in a data center fabric; spine switches often need higher port count than top of rack devices, fat tree designs force the designer into choosing between multiple, less expensive, fixed configuration devices or more consistent usage of more expensive, blade- or slot-based devices, etc.

统一控制管理

可以从多个供应商购买设备，然后依赖控制平面和管理接口的开放标准实现。例如，网络运营商可能决定标准化 IS-IS 作为整个网络的控制平面，并标准化基于开放标准的 YANG 模型，使用 NETCONF 或 RESTCONF 传输来管理所有设备。

It is possible to purchase equipment from multiple vendors, and then rely on open standards implementations of control planes and management interfaces. For instance, a network operator may decide to standardize on IS-IS as a control plane throughout their network, and to standardize on open standards based YANG models, using a NETCONF or RESTCONF transport to manage all the devices.

当然，在尝试部署此类解决方案时需要考虑一些权衡。例如：

There are, of course, several tradeoffs to consider when attempting to deploy this type of solution. For instance:

• 供应商受财务驱动而支持功能，而不是管理界面。这意味着管理界面通常远远落后于网络产品新协议或功能的实施。

• Vendors are financially driven to support features, rather than management interfaces. This means management interfaces often lag far behind the implementation of new protocols or features on networking products.

• 至少在某种程度上，供应商不再支持标准化接口。通过支持标准化接口，供应商可以在任何部署中轻松更换——实际上，标准化接口使设备变得更加商品化。

• Vendors are, at least to some degree, driven away from supporting standardized interfaces. By supporting standardized interfaces, vendors are opening themselves up to being easily replaced in any deployment—in effect, standardized interfaces drive equipment into more of a commodity role.

• 标准机构通常在开发新功能或协议扩展方面进展缓慢。如果网络运营商等到新想法完全标准化，或者甚至看起来将以特定形式标准化，他们可能会面临失去竞争对手优势的风险。

• Standard bodies are often slow to produce new features or extensions to protocols. If a network operator waits until a new idea is fully standardized, or even looks like it will be standardized in a specific form, they could risk losing some advantage over their competitors.

当然，通常可以通过各种工具来掩盖接口差异——网络管理系统和像 Puppet 这样的开源工具通常可以通过 thunk 或硬件抽象层来找到一组通用的功能。然而，在使用这些类型的工具时，请记住抽象泄漏的问题。

It is often possible to cover up interface differences through a variety of tools, of course—network management systems and open source tools like Puppet can often work to find a common set of features through a thunk or hardware abstraction layer. However, keep the problem of leaky abstractions in mind when working with these sorts of tools.

抽象泄漏法则

The Law of Leaky Abstractions

什么是泄漏抽象？2002 年，Joel Spolsky在他的博客Joel on Software上创造了泄漏抽象定律。¹泄漏抽象定律假设：“所有不平凡的抽象，在某种程度上，都会泄漏。” Joel 使用 TCP 作为此类泄漏抽象的示例 — TCP 实际上是底层 IP 连接的抽象，旨在模拟跨不可靠路径的面向连接的端到端链路。问题在于 TCP 作为一个抽象概念存在泄漏。

What is a leaky abstraction? In 2002, Joel Spolsky coined the law of leaky abstractions on his blog, Joel on Software.¹ The law of leaky abstractions posits, “All non-trivial abstractions, to some degree, leak.” Joel uses TCP as an example of such a leaky abstraction—TCP is, in effect, an abstraction of the underlying IP connectivity designed to emulate a connection oriented end-to-end link across an unreliable path. The problem is that TCP, as an abstraction, leaks.

1 . Joel Spolsky，“泄漏抽象定律”，博客，Joel on Software，np，最后修改时间为 2002 年 11 月 11 日，访问日期为 2015 年 1 月 19 日，http://www.joelonsoftware.com/articles/LeakyAbstractions.html。

1. Joel Spolsky, “The Law of Leaky Abstractions,” Blog, Joel on Software, n.p., last modified November 11, 2002, accessed January 19, 2015, http://www.joelonsoftware.com/articles/LeakyAbstractions.html.

有两种方法可以看到这种泄漏效果。首先是要认识到，无论 TCP 用表面上面向连接的链路覆盖底层网络的效果如何，仍然存在底层无连接、不可靠链路的伪影需要处理。例如，当使用 TCP 传输数据的软件遇到抖动时，它必须决定要做什么 - 是否缓冲流量，以便向用户提供他们期望从端到端无损连接中获得的体验，以中断连接，然后重新开始，或者部署一些其他策略。当然，所选择的具体策略取决于应用程序，但应用程序不能像 TCP 连接真正是两个主机或两个设备之间的端到端连接一样运行。

There are two ways to see this leaking effect. The first is to realize that no matter how well TCP covers up the underlying network with an apparently connection oriented link, there are still artifacts of the underlying connectionless, unreliable link to deal with. When a piece of software using TCP to transfer data encounters jitter, for instance, it must decide what to do—whether to buffer the traffic so the user is presented with the experience they expect from an end-to-end lossless connection, to break the connection, and start over, or to deploy some other strategy. The specific strategy chosen is application dependent, of course, but the application cannot operate as if a TCP connection is truly an end-to-end connection between two hosts, or two devices. Thus the application developer must understand something of the IP network over which the TCP session runs—so the abstraction leaks.

查看 TCP 中抽象泄漏的第二种方法是考虑当 TCP 会话实际失败时会发生什么。如果 TCP 确实是端到端连接，那么解决问题的工程师可以将自己限制在 TCP 机制上——计时器、窗口、数据编组以及 TCP 用于传输信息的其他技术。然而，实际上，研究 TCP 问题的工程师必须深入研究 TCP 和 IP 之间的交互、IP 本身、IP 和较低物理层之间的交互、转发 IP 流量的设备上使用的排队机制、指导 IP 流量的控制平面。 IP 流量等。抽象泄漏——仅仅因为它看起来像端到端无损连接并不意味着您实际上可以一直这样对待它。关键是要知道什么时候可以将 TCP 视为端到端无损连接，

The second way to see the leaky abstraction in TCP is to consider what happens when a TCP session actually does fail. If TCP were truly an end-to-end connection, an engineer troubleshooting the problem could confine themselves to TCP mechanisms—the timers, windows, data marshaling, and other techniques TCP uses to transfer information. In reality, however, engineers working on TCP problems must dive into the interaction between TCP and IP, IP itself, the interaction between IP and the lower physical layers, the queuing mechanisms used on the devices forwarding the IP traffic, the control plane that directs IP traffic, etc. The abstraction leaks—just because it looks like an end-to-end lossless connection doesn’t mean you can actually treat it that way all the time. The key is to know when you can treat TCP as an end-to-end lossless connection, and when you can’t, and to manage the exceptions.

同样的问题也出现在网络管理中，特别是在设备级别。您可以构建一个抽象来描述所有不同的配置可能性、所有不同的操作模式等，对于任何给定设备。然而，我们面临两个问题。首先，建立一个完整的列表就是破坏抽象本身的意义。其次，无论这种抽象多么完整，它仍然会存在漏洞。有时，工程师仍然必须通过抽象来理解真实设备上的真实配置是什么，以及如何操纵它以使设备执行工程师需要它执行的操作。

The same problem crops up in network management, particularly at the device level. You can build an abstraction that will describe all the various configuration possibilities, all the various modes of operation, etc., for any given device. However, there are two problems that confront us. First, to build a complete list is to destroy the point of the abstraction itself. Second, no matter how complete such an abstraction might be, it will still be leaky. There will still be times when the engineer must work through the abstraction to understand what the real configuration on the real device is, and how to manipulate it to make the device do what the engineer needs it to do.

泄漏抽象的问题在于，抽象越泄漏，抽象的实际用处就越小。例如，在某些时候，弄清楚如何通过设备模型（例如 YANG）配置设备比简单地使用本机配置命令直接配置设备变得更加困难。当然，这是一个纪律问题，只要有可能就使用抽象，并将工作投入到修复抽象中，而不是简单地定期绕过它。但最终，保持抽象“干净”所需的时间与简单地完成工作之间必须保持某种平衡。如果抽象的漏洞如此之多，以至于需要花费大量的时间和精力来处理它，那么最终的结果可能会变得更加复杂，而不是更少。

The problem with leaky abstractions is that the leakier the abstraction, the less useful the abstraction really is. At some point, for instance, it becomes harder to figure out how to configure the device through a device model (such as YANG) than to simply configure the device directly using native configuration commands. It’s a matter of discipline, of course, to use the abstraction whenever possible, and to put the work into fixing the abstraction rather than simply going around it on a regular basis. In the end, though, there has to be some balance between the amount of time it takes to keep the abstraction “clean,” and simply getting the work done. If the abstraction is so leaky that it requires a lot of time and effort to work within it, the end result might be more complexity rather than less.

统一运输

网络复杂性中被忽视的领域之一是现实世界中部署的各种传输系统。还记得旧的以 IP 为中心的模型吗？一种传输通过多种电路，并且在顶部运行许多不同的应用程序？今天的现实是覆盖、隧道类型和传输散布在整个网络中的意大利面条，如图6.2（部分）所示。

One of the overlooked areas of network complexity is the wide array of transport systems deployed in the real world. Remember the old IP focused model, with one transport over a variety of circuits, and a lot of different applications running on top? The reality today is a spaghetti of overlays, tunnel types, and transports scattered throughout the network, as Figure 6.2 (partially) illustrates.

图 6.2 意大利面条运输系统

Figure 6.2 The Spaghetti Transport System

意大利面条运输系统有几个有趣（且难以管理）的功能，包括：

The spaghetti transport system has several interesting (and difficult to manage) features, including:

• 一些上层协议严重依赖于下层协议中包含的信息和生成的状态。例如，许多上层应用程序实际上依赖 IP 地址作为连接到网络的特定系统的一种标识符，尽管 IP 地址实际上是一个定位符。

• Several upper layer protocols are heavily reliant on information contained in and state generated by lower layer protocols. For instance, many upper layer applications actually rely on the IP address as a sort of identifier for a particular system connected to the network, although the IP address is actually a locator.

• 多个上层协议可以承载下层协议，从而创建复杂且重叠的堆栈。尽管虚拟可扩展局域网（VXLAN）隧道依赖于IP，但它实际上承载的是以太网帧；IPv4、IPv6 和通用路由封装 (GRE) 隧道（包括 GRE 隧道内承载的 MPLS）通常承载在 VXLAN 之上。

• Several upper layer protocols can carry lower layer protocols, creating a complex and overlapping stack. Although Virtual Extensible Local Area Network (VXLAN) tunnels rely on IP, it actually carries Ethernet frames; IPv4, IPv6, and Generic Routing Encapsulation (GRE) tunnels (including MPLS carried inside a GRE tunnel) are regularly carried on top of VXLAN.

• IPv4 和IPv6 可以作为并行网络协议运行，可以使用自己的控制平面或单独的控制平面。

• IPv4 and IPv6 can run as parallel network protocols, either with their own control plane or separate control planes.

这些协议中的每一个都是一个有漏洞的抽象；以这种方式将它们一个一个地放在另一个之上，特别是当协议设计和部署在“堆叠循环”中时，例如以太网上的 VXLAN 上的 IP 上的以太网，会导致每个抽象中的泄漏混合在一起，从而在管理和故障排除。该图中的每对协议也代表整个系统中的另一个交互界面。

Each of these protocols is a leaky abstraction; laying them one on top of another in this way, especially when protocols are designed and deployed in “stacking loops,” such as Ethernet over VXLAN over IP over Ethernet, causes the leaks in each abstraction to mingle, making a complete mess in terms of management and troubleshooting. Each pair of protocols in this illustration represent another interaction surface in the overall system, as well.

可以采取什么措施来缓解意大利面条运输系统的问题？将任何给定网络中运行的传输数量减少到尽可能少。设计人员应尝试选择最少数量的协议（包括覆盖层和底层），以支持所有应用程序和所有要求。这必然涉及对特定应用程序或系统的支持的权衡，但降低复杂性所带来的好处是非常值得的，因为它将减少 MTTR、控制平面状态和许多其他因素。以 MPLS 为例，单个 MPLS 部署可以支持以太网、IPv4 和 IPv6 虚拟通过非常小的传输系统与完全虚拟化进行链接。对于同一组问题，部署双栈 IPv4/IPv6 和 VXLAN 进行第 2 层虚拟化（在 IP 之上进行第 3 层虚拟化）是一种更为复杂的解决方案。

What can be done to alleviate this spaghetti transport system? Reducing the number of transports running in any given network to the minimum possible. Designers should try and choose the minimal number of protocols, both overlays and underlays, which will support all the applications and all the requirements. This will necessarily involve tradeoffs in support for specific applications or systems, but the gain in reduced complexity is well worth it, as it will reduce the MTTR, control plane state, and many other factors. Using MPLS as an example—a single MPLS deployment can support Ethernet, IPv4, and IPv6 virtual links with full virtualization across a very minimal transport system. Deploying dual-stack IPv4/IPv6 and VXLAN for Layer 2 virtualization (with IP on top for Layer 3 virtualization) is a much more complex solution to the same set of problems.

请记住，仅仅因为另一个协议可用，并不意味着您必须将其部署在您的网络中 - 仅仅因为您可以，并不意味着您应该这样做。

Remember that just because another protocol is available, it doesn’t mean you have to deploy it in your network—just because you can doesn’t mean you should.

仅对几种传输进行标准化的一个缺点可以通过返回 MPLS 来说明。在企业或数据中心级设备中获得 MPLS 支持要困难得多（或者更昂贵），因为 MPLS 被视为“传输服务提供商解决方案”。有时，我们自己的先入之见可能会导致我们根据解决方案的“用途”来选择解决方案，而不是根据它们真正有用的用途；最终，这会使我们的网络变得更加复杂，而不是更加复杂。

The one downside to standardizing on just a few transports can be illustrated by returning to MPLS. It’s much harder (or rather more expensive) to obtain MPLS support in enterprise or data center class equipment, as MPLS is seen as a “transit service provider solution.” Sometimes our own preconceptions can cause us to choose solutions based on what they’re “meant for,” rather than what they’re really useful for; in the end, this can make our networks more, rather than less, complex.

可互换模块

Interchangeable Modules

在设备级别，使用单一供应商的产品，或者以更完整的方式使用相同型号的设备（请参阅前面有关数据中心结构的示例），可以通过一致性来降低复杂性。相同的原理也可以应用在网络级别，方法是对网络中每种不同类型的模块进行编目，然后将每个模块构建得尽可能相似。例如，您可以将网络划分为一小组拓扑：

At the device level, using a single vendor’s products, or in a more complete way, the same model of device (see the previous example on data center fabrics), can reduce complexity through uniformity. The same principle can be applied at the network level, as well, by cataloging each of the different types of modules in the network, and then building each one to be as similar as possible. For instance, you might be able to divide a network into a small set of topologies:

• 校园

• Campus

• 数据中心

• Data Center

• 存在点

• Point of Presence

• 核

• Core

对于每个“位置”，确定一组角色以及与该角色相关的要求。对于每个引脚中的每个角色，部署一个支持各种要求的网络拓扑和配置。只要在以下方面保持纪律：

For each one of these “places,” determine a set of roles and the requirements that go with that role. For each role in each pin, deploy a single network topology and configuration that will support the range of requirements. So long as discipline is maintained in:

• 保持单个“地点”的每个实例的配置和部署在时间和地点上都相同。

• Keeping the configurations and deployments of every instance of a single “place” the same across time and location.

• 将“地点”和角色的数量保持在尽可能的最低限度。

• Keeping the number of “places” and roles to the minimum possible.

设计、部署、管理和排除这些模块故障所需的工作量大大减少。

The amount of work required to design, deploy, manage, and troubleshoot these modules is greatly diminished.

在数据中心内，这种思维方式也经常用于“pod”或“模块”。数据中心内的每组机架都可以被视为一个模块化单元，其中数据中心以单元（而不是设备）的形式增长，并且单元（而不是单个设备）被升级或更换。像往常一样，可互换模块会带来许多问题。

Within the data center, this type of thinking is often used of “pods,” or “modules,” as well. Each set of racks within a data center can be considered a modular unit, where the data center grows in units (rather than devices), and units are upgraded or replaced, rather than individual devices. There are, as usual, a number of problems that come along with interchangeable modules.

首先，可互换零件通常不是。尽管网络管理者做了很多工作来使每个模块相同，但本地条件、跨时间分布的情况都违背了一致性。当制造商更换设备线或将一个设备替换为另一个设备时，确保大量相同模块的一致性很快就会变得困难。当互连网络中的不同位置 (PIN) 时，这也是一个问题。例如，如果您决定升级 PIN 之间的连接速度，则需要修改整个网络中的每个 PIN。面对此类问题，模块化通常很难维护。

First, interchangeable parts often aren’t. In spite of all the work network managers do to make each module identical, local conditions, spread across time, work against consistency. As manufacturers replace lines of equipment, or replace one device with another, it quickly becomes difficult to ensure consistency across a large number of identical modules. This is also a problem when interconnecting different Places In the Network (PINs). As an example, if you decide to upgrade the connection speed between PINs, you will need to modify every PIN in the entire network. Modularity is often difficult to maintain in the face of such problems.

其次，经常隐藏在可互换模块背后的一个问题是“网络中的位置”综合症，其中模块成为所有网络设计和操作的焦点，从而使网络作为一个整体或作为一个系统被排除在外。予以考虑。这有点像建造一座房子，选择许多你认为你想要的不同房间，然后选择一条走廊将它们连接起来。从住在房子里的人的角度来看，这可能真的很好，但建筑商在尝试建造客户想要的东西时将面临一些真正的麻烦，而且维护将是一场灾难。

Second, a problem often hidden behind interchangeable modules is the “places in the network” syndrome, where the modules become the focus of all network design and operations, leaving the network as a whole, or as a system, off the table as something to be considered. It’s a bit like building a house by choosing a lot of different rooms you think you want, and then choosing a hallway to connect them all. It might be really nice from the perspective of the person living in the house, but the builder is going to face some real trouble in trying to build what the customer wants—and maintenance is going to be a disaster.

笔记

Note

有关 PIN 的更完整说明，请参阅本章后面的“网络中的位置”部分。

See the section on “Places in the Network” later in this chapter for a more complete description of PINs.

模块化如何解决复杂性问题

How Modularity Attacks the Complexity Problem

网络工程师经常将模块化与分层网络设计和聚合等同起来，但这两者并不完全相同。分层网络设计是一门以模块化为原则的学科，但并非所有分层网络设计都是模块化的，也不是所有模块化网络设计都是分层的。同样，聚合通常取决于模块化，但您可以构建一个非常模块化的网络，根本不聚合任何信息。那么，如果聚合和层次结构不是重点，那么模块化以及此处给出的示例（统一硬件、统一管理、统一传输和可互换模块）如何真正影响网络设计的复杂性？

Network engineers often equate modularity with hierarchical network design and aggregation, but these two are not precisely the same thing. Hierarchical network design is a discipline that uses modularity as one of its principles, but not all hierarchical network designs are modular, nor are all modular network designs hierarchical. In the same way, aggregation often depends on modularity, but you can build a very modular network that doesn’t aggregate any information at all. So, if aggregation and hierarchy are not the point, how does modularity, and the examples given here—uniform hardware, uniform management, uniform transport, and interchangeable modules—really impact the complexity of a network’s design?

虽然真正的模块化设计通常能够减少状态，但模块化设计本身实际上并不能减少状态。对于速度也是如此——构建模块化设计可能会提供降低信息通过控制平面传播的速度的机会，但它不会直接影响网络运行的速度。

While a truly modular design often leads to the ability to reduce state, modular designs, themselves, don’t actually reduce state. The same can be said for speed, as well—building a modular design might provide opportunities to reduce the speed at which information is spread through the control plane, it doesn’t directly impact the speed at which the network operates.

因此， Surface是模块化降低网络设计复杂性的主要手段。通过将网络分解为多个较小的部分，模块化减少了网络中交互表面的大小，如图6.3所示。

Surface, then, is the primary means through which modularity reduces complexity in network design. By breaking the network up into multiple smaller pieces, modularity reduces the size of interaction surfaces in the network, as illustrated in Figure 6.3.

图 6.3 模块化和交互界面

Figure 6.3 Modularization and Interaction Surfaces

信息隐藏

Information Hiding

从管理和控制平面的角度来看，模块化本身通过减小交互表面的大小，在一定程度上降低了网络复杂性。但是，如上所述，模块化并没有真正解决网络复杂性的状态或速度方面。然而，模块化确实可以实现信息隐藏，而信息隐藏会直接攻击状态和速度。让我们看一下网络设计中常用的两种信息隐藏形式：聚合和虚拟化。

Modularization, on its own, provides some reduction in network complexity, both from a management and a control plane perspective, by reducing the size of the interaction surfaces. But, as above, modularization doesn’t really attack the state or speed aspects of network complexity. Modularity does, however, enable information hiding—and information hiding directly attacks state and speed. Let’s look at two forms of information hiding commonly used in network design: aggregation and virtualization.

聚合

Aggregation

到目前为止，聚合是网络中最常见的信息隐藏形式。实际上，网络设计中使用了两种类型的聚合，尽管它们经常被混为一谈：隐藏拓扑信息和总结可达性信息。了解两者之间差异的最简单方法是检查链路状态协议（例如 IS-IS）中的聚合。图 6.4将用于说明差异。

Aggregation is, by far, the most common form of information hiding in networks. There are actually two types of aggregation in use in network design, although they are often conflated into the same thing: hiding topology information and summarizing reachability information. The easiest way to understand the difference between the two is by examining aggregation in a link state protocol, such as IS-IS. Figure 6.4 will be used to illustrate the difference.

图 6.4 聚合

Figure 6.4 Aggregation

给定如图所示的网络，除了将路由器 A 和 B 放置在单个 2 级泛洪域中，并将路由器 B、C、D 和 E 放置在单独的 1 级泛洪域中之外，无需任何配置，路由器 B 在其 1 级中会看到什么IS-IS 拓扑数据库与路由器 A 在其 2 级拓扑数据库中看到的内容？

Given the network as illustrated, with no configurations beyond placing Routers A and B in a single level 2 flooding domain, and Routers B, C, D, and E in a separate level 1 flooding domain, what would Router B see in its level 1 IS-IS topology database versus what Router A sees in it level 2 topology database?

路由器B：

Router B:

• 2001:db8:0:1::/64 连接到 D

• 2001:db8:0:1::/64 connected to D

• 2001:db8:0:2::/64 连接到 E

• 2001:db8:0:2::/64 connected to E

• 路由器 B => 路由器 D

• Router B => Router D

• 路由器 D => 路由器 B

• Router D => Router B

• 路由器E => 路由器D

• Router E => Router D

• 路由器D => 路由器E

• Router D => Router E

• 路由器E => 路由器C

• Router E => Router C

• 路由器C => 路由器E

• Router C => Router E

• 路由器C => 路由器B

• Router C => Router B

• 路由器 B => 路由器 C

• Router B => Router C

• 路由器 B 连接到 2 级路由域（路由器 B 的 LSP 中设置的连接位）

• Router B connected to a level 2 routing domain (the connected bit set in Router B’s LSP)

路由器A：

Router A:

• 可通过路由器 B 访问 2001:db8:0:1::/64

• 2001:db8:0:1::/64 reachable through Router B

• 可通过路由器 B 访问 2001:db8:0:2::/64

• 2001:db8:0:2::/64 reachable through Router B

• 路由器A => 路由器B

• Router A => Router B

• 路由器 B => 路由器 A

• Router B => Router A

检查这两个列表，最大的区别如下：

Examining these two lists, the big differences are as follows:

• 两个可达子网的连接点都附在Router B 的路由信息上，而不是附在Router D 和E 上。

• The connection points for the two reachable subnets are attached to Router B’s routing information, rather than Routers D and E.

• 路由器B、C、D 和E 之间的链路均从拓扑数据库中删除。

• The links between Routers B, C, D, and E are all removed from the topology database.

通过将 1 级泛洪域内的所有可达目的地附加到 1 级/2 级泛洪域边界，Router A 不需要了解 1 级拓扑的内部详细信息。在像IS-IS这样的链路状态协议中创建多个泛洪域的目的实际上是阻止拓扑信息从“外围”泛洪域进入“核心”泛洪域——从1级泛洪域进入2级泛洪域。泛洪域。然后，路由器 B 聚合拓扑信息，仅携带可到达的目的地和到达这些目的地的成本，而不是将 1 级洪泛域内每对路由器之间的链路状态传递给路由器 A。聚合此拓扑信息有何作用完成？两件事情：

By attaching all the reachable destinations within the level 1 flooding domain to the level 1/level 2 flooding domain border, Router A doesn’t need to know the internal details of the level 1 topology. The point of creating multiple flooding domains in a link state protocol like IS-IS is, in fact, to block the topology information from “outlying” flooding domains into the “core” flooding domain—from the level 1 flooding domains into the level 2 flooding domain. Router B, then, is aggregating the topology information, carrying just the reachable destinations and the cost to reach them, rather than passing the link state between each pair of routers within the level 1 flooding domain to Router A. What does aggregating this topology information accomplish? Two things:

拓扑信息的聚合通过从级别 2 泛洪域中删除级别 1 泛洪域中承载的链路信息，减少了控制平面中承载的状态，从而减小了拓扑数据库的大小以及 SPF 算法必须的树的大小计算跨度。

• Aggregation of topology information decreases the state carried in the control plane by removing the link information carried in the level 1 flooding domain from the level 2 flooding domain, hence reducing the size of the topology database and the size of the tree the SPF algorithm must calculate across.

• 拓扑信息的聚合通过从二级数据库中删除有关一级泛洪域内链路状态的信息，降低了信息流经控制平面的速度。它例如，对于路由器 A 来说，路由器 D 和 E 之间的链路状态是什么并不重要，只要从路由器 B 的角度来看，可达目的地保持不变即可。通过阻止拓扑信息流入第 2 级链路状态拓扑数据库、聚合会减慢更新速度，从而减慢路由器 A 对这些更新做出反应的速度。

• Aggregation of topology information decreases the speed of the information flowing through the control plane by removing information about the state of the links within the level 1 flooding domain from the level 2 database; it doesn’t matter to Router A what the state of the link between Routers D and E are, for instance, so long as the reachable destinations remain constant from the perspective of Router B. By blocking topology information from flowing into the level 2 link state topology database, aggregation slows down the pace of updates, and hence the pace at which Router A must react to those updates.

通过将路由器 B 配置为聚合其所连接的 1 级泛洪域中的可达性信息，可以进一步进行聚合。在这种情况下，两个路由 2001:db8:0:1::/64 和 2001:db8:0:2::/64 可以聚合到一个较短的前缀路由 2001:db8::0/61 。如果您在配置此聚合后检查路由器 A 的 2 级拓扑数据库，您会发现它现在稍微小一些。

Aggregation can be taken one step further by configuring Router B to aggregate the reachability information in the level 1 flooding domain to which it’s attached. In this case, the two routes, 2001:db8:0:1::/64 and 2001:db8:0:2::/64, can be aggregated to a single shorter prefix route, 2001:db8::0/61. If you examined Router A’s level 2 topology database after this aggregation is configured, you’d find it is now slightly smaller.

• 可通过路由器 B 访问 2001:db8::0/64

• 2001:db8::0/64 reachable through Router B

• 路由器A 连接到路由器B

• Router A connected to Router B

• 路由器 B 连接到路由器 A

• Router B connected to Router A

再次从状态和速度的角度来审视这一点：

Once again, examining this from the perspective of state and speed:

• 可达性信息的聚合通过将两个可达目的地减少为单个可达目的地来减少控制平面中承载的状态。

• Aggregation of reachability information decreases the state carried in the control plane by reducing the two reachable destinations to a single reachable destination.

• 拓扑信息的聚合通过删除有关 1 级洪泛域内两个单独可达目的地的状态的信息，降低了信息流经控制平面的速度。无论 2001:db8:0:1::/64 或 2001:db8:0:2::/64 的状态如何，2001:db8::0/61 的状态保持不变。

• Aggregation of topology information decreases the speed of the information flowing through the control plane by removing information about the state of the two individual reachable destinations within the level 1 flooding domain. No matter what the state of either 2001:db8:0:1::/64 or 2001:db8:0:2::/64 are, the state of 2001:db8::0/61 remains constant.

与所有抽象一样，聚合也有一系列权衡（一如既往，天下没有免费的午餐）。上一章花了很多时间讨论聚合对拉伸的影响——这里不再重复讨论，但记住这一点很重要。然而，除了延伸之外，网络工程师还应该记住，聚合还受到抽象泄漏定律的约束，如上面的侧边栏所述。聚集体如何泄漏？图 6.5将聚合展示为一种有漏洞的抽象。

Aggregation, like all abstractions, has a set of tradeoffs (as always, there is no such thing as a free lunch). The last chapter spent a good deal of time discussing the impact of aggregation on stretch—that discussion won’t be repeated here, but it’s important to bear in mind. Beyond stretch, however, network engineers should remember that aggregation is also subject to the law of leaky abstractions, explained in a sidebar above. How do aggregates leak? Figure 6.5 illustrates aggregation as a leaky abstraction.

图 6.5 作为泄漏抽象的聚合

Figure 6.5 Aggregation as a Leaky Abstraction

路由器 B 配置为使用两个组件路由中的最低度量将 2001:db8:0:1::/64 和 2001:db8:0:2::/64 聚合到 2001:db8::/61。在这种情况下，度量将从 [B,D] 路径中选择，因此聚合将为通告的度量为 2。但是，如果路由器 B 和 D 之间的链路发生故障，则组件中最低的度量将是路径 [B,C,E]，总度量为 3。当该链路发生故障时，聚合路由会将度量从成本 2 更改为成本 3，因此抽象通过聚合泄漏了有关拓扑的信息。当然，堵住这个漏洞是可能的。然而，重要的是要记住，这里使用的任何解决方案都需要考虑其自身的一系列权衡。

Router B is configured to aggregate 2001:db8:0:1::/64 and 2001:db8:0:2::/64 to 2001:db8::/61 using the lowest metric from the two component routes. In this case, the metric would be chosen from the [B,D] path, so the aggregate would be advertised with a metric of 2. If the link between Router B and D fails, however, the lowest metric among the components would be the path [B,C,E], with a total metric of 3. When this link fails, the aggregate route will change metric from a cost of 2 to a cost of 3—hence the abstraction has leaked information about the topology through the aggregation. It is possible, of course to plug this leak; however, it’s important to remember that any solution used here will have its own set of tradeoffs to consider.

故障域和信息隐藏

Failure Domains and Information Hiding

如果不对故障域进行一些检查，对信息隐藏的讨论就不完整。聚合和虚拟化都用于限制故障域的大小，但什么是故障域？它与复杂性有何关系？首先，我们来定义什么是故障域，然后看看故障域和网络复杂性之间的关系。图 6.6说明了故障域。

No discussion of information hiding would be complete without some examination of failure domains. Both aggregation and virtualization are used to limit the size of a failure domain—but what is a failure domain, and how does it relate to complexity? To begin, let’s define what a failure domain is, then let’s look at the relationship between failure domains and network complexity. Figure 6.6 illustrates failure domains.

图 6.6 故障域

Figure 6.6 Failure Domains

该 IS-IS 网络中的路由器 B 配置为将 2001:db8:0:1::/64 和 2001:db8:0:2::/64 聚合成一条路由 2001:db8::/61，即然后通告到 2 级泛洪域（朝向路由器 C）。2001:db8:0:8::/64 不属于此聚合范围，因此它“按原样”发布到第二级泛洪域。使用此配置，您可以构建网络中所选路由器的数据库与这三个路由相关的高级概述。

Router B in this IS-IS network is configured to aggregate 2001:db8:0:1::/64 and 2001:db8:0:2::/64 into a single route, 2001:db8::/61, which is then advertised into the level 2 flooding domain (toward Router C). 2001:db8:0:8::/64 does not fall within this aggregate, so it is advertised “as is” into the level two flooding domain. Using this configuration, you can build a high-level overview of what the database for a selection of routers in the network will look like in relation to these three routes.

•路由器 B，1 级数据库：

• Router B, Level 1 Database:

• 路由器B 连接到路由器A。

• Router B is connected to Router A.

• RouterB 连接到2 级泛洪域。

• Router B is connected to a level 2 flooding domain.

• 路由器A 连接到路由器B。

• Router A is connected to Router B.

• 路由器A 连接到2001:db8::0/64。

• Router A is connected to 2001:db8::0/64.

• 路由器A 连接到2001:db8:0:2::/64。

• Router A is connected to 2001:db8:0:2::/64.

• 路由器A 连接到2001:db8:0:8::/64。

• Router A is connected to 2001:db8:0:8::/64.

•路由器 B、C 和 D，2 级数据库：

• Routers B, C, & D, Level 2 Database:

• 路由器B 连接到路由器C。

• Router B is connected to Router C.

• 路由器C 连接到路由器B。

• Router C is connected to Router B.

• 路由器C 连接到路由器D。

• Router C is connected to Router D.

• 路由器D 连接到路由器C。

• Router D is connected to Router C.

• 路由器B 连接到2001:db8::/61。

• Router B is connected to 2001:db8::/61.

• 路由器B 连接到2001:db8:0:8::/64。

• Router B is connected to 2001:db8:0:8::/64.

•路由器E：

• Router E:

• 路由器D 连接到路由器E。

• Router D is connected to Router E.

• 路由器E 连接到路由器D。

• Router E is connected to Router D.

• RouterD 连接到2 级泛洪域。

• Router D is connected to a level 2 flooding domain.

检查网络中各个位置的链路状态数据库的高级视图将显示以下内容：

Examining this high-level view of the link state databases at various places in the network would show the following:

• 如果2001:db8:0:1::/64 或2001:db8:0:2::/64 与Router A 断开，则只有Router A 和B 需要重新计算其SPT。

• If 2001:db8:0:1::/64 or 2001:db8:0:2::/64 are disconnected from Router A, only Routers A and B will need to recalculate their SPT.

• 如果2001:db8:0:8::/64 与路由器A 断开连接，则路由器B、C 和D 将需要重新计算其SPT。

• If 2001:db8:0:8::/64 is disconnected from Router A, Routers B, C, and D will need to recalculate their SPT.

因此，隐藏信息可以减少必须重新计算其 SPT 以响应网络拓扑变化的路由器数量。事实上，这是您可能会发现的故障域的最佳定义：

Hiding information, then, reduces the number of routers that must recalculate their SPT in reaction to a change in the network topology. This is, in fact, as good a definition of a failure domain as you are likely to find:

故障域是当网络拓扑或可达性发生变化时必须与控制平面交互的一组设备。

A failure domain is the set of devices that must interact with the control plane when network topology or reachability changes.

根据此定义，如果 2001:db8:0:2::/64 与路由器 A 断开连接，则故障域仅包含路由器 A 和 B。在这种情况下，路由器 C 到 E 不包含在故障域中，因为该信息在他们的控制平面中并没有改变。因此，通过聚合隐藏信息，可以减小故障域的大小。这缩小了控制平面内交互表面的范围，并降低了控制平面接收新信息的速度。

Given this definition, if 2001:db8:0:2::/64 is disconnected from Router A, the failure domain contains only Routers A and B. Routers C through E are not included in the failure domain in this case, because the information in their control planes doesn’t change. So by hiding information through aggregation, the size of the failure domain has been reduced. This reduces the scope of the interaction surfaces within the control plane, as well as reducing the speed at which the control plane receives new information.

此示例中需要注意的另一点是，故障域不是您可以在网络的任何特定部分周围绘制的“实线”。一些信息（例如上面给出的聚合度量变化的示例）总是会通过网络中隐藏信息的任何点泄漏（请参阅泄漏抽象定律的前边栏），因此实际上存在许多不同的重叠故障任何给定网络中的域。

Another point to note from this example is that the failure domain is not a “solid line” you can paint around any particular part of the network. Some information (such as the example of the aggregate’s metric changing given above) will always leak through any point where information is being hidden in the network (see the previous sidebar on The Law of Leaky Abstractions), so there are actually many different overlapping failure domains in any given network.

关于信息隐藏的最终想法

Final Thoughts on Information Hiding

以一种或另一种形式隐藏信息是网络设计者应对复杂性的最佳工具之一。通过隐藏模块之间的信息：

Information hiding in one form or another is one of the best tools the network designer has to deal with complexity. By hiding information between modules:

• 控制平面中携带的状态量减少。聚合消除了可达性和拓扑信息，从而减少了控制平面中承载的信息总量。虚拟化（在上一章中讨论）将拓扑和可达性信息分解在多个控制平面之间，以便每个控制平面仅管理总网络状态的一个子集。

• The amount of state carried in the control plane is reduced. Aggregation removes reachability and topology information, reducing the total amount of information carried in the control plane. Virtualization (discussed in the last chapter) breaks the topology and reachability information up among multiple control planes, so that each control plane only manages a subset of the total network state.

•控制平面的速度降低。例如，聚合通过完全删除状态（拓扑）或用易失性较小的状态替换易失性较高的状态（可达性信息的汇总）来降低控制平面承载信息的速度。虚拟化通过将更改分散到多个控制平面来降低控制平面的更改速度（重要的是要记住，虚拟化对控制平面操作速度的影响小于聚合）。

• The speed of the control plane is reduced. Aggregation, for instance, reduces the speed at which the information the control plane is carrying by either removing state entirely (topology) or replacing more volatile state with less volatile state (summarization of reachability information). Virtualization reduces the speed of changes in the control plane by spreading the changes across multiple control planes (it’s important to remember that virtualization impacts the speed of control plane operations less than aggregation does).

• 交互面主要通过模块化包含在聚合中，并在网络中创建“阻塞点”，限制控制平面各个部分交互的位置。虚拟化通过将特定应用程序或客户绑定到特定逻辑拓扑来减小交互界面的大小，从而允许将每组应用程序视为独立于网络上运行的其余应用程序的情况。当然，虚拟化并不像聚合那么简单，因为虚拟拓扑本身必须交互，从而在网络中创建另一个交互表面。

• The interaction surfaces are contained in aggregation primarily through modularization, and the creation of “choke points” in the network limiting the places where various pieces of the control plane interact. Virtualization reduces the size of the interaction surfaces by tying specific applications or customer to specific logical topologies, thus allowing each set of applications to be treated as a case that’s independent of the rest of the applications running on the network. Of course, virtualization isn’t quite as straight forward as aggregation, because the virtual topologies themselves must interact, creating another interaction surface in the network.

楷模

Models

模型不是处理复杂性的“网络”工具，而是一种对网络中发生的情况进行分类和抽象的方法。大多数网络工程师通常熟悉七层和四层模型，这些是抽象协议操作的好模型。对于理解整个网络的部署和操作，或者网络和应用程序的操作，有用的模型怎么样？本节介绍了三种不同的模型，最后介绍了一种建模语言。

Models aren’t an “on network” tool to deal with complexity, but rather a way of categorizing and abstracting out what is happening in the network. Most network engineers are generally familiar with the seven and four layer models—and these are good models for abstracting the operation of protocols. What about useful models for understanding the deployment and operation of the network as a whole, or the operation of the network and applications? Three different models are presented in this section, and finally a modeling language.

瀑布

Waterfall

瀑布模型不是网络运行的模型，而是流量的模型。该模型基于这样的基本见解：所有路由和交换协议本质上都会构建一棵源于目的地的树，然后分布到每个可用的源。图 6.7显示了瀑布模型。

The waterfall model isn’t a model of network operation, but rather a model of traffic flows. This model is based on the basic insight that all routing and switching protocols essentially build a tree sourced at the destination, and spread out to each available source. Figure 6.7 shows the waterfall model.

图 6.7 瀑布模型

Figure 6.7 The Waterfall Model

瀑布模型显示了数据流如何在每个网络设备上在网络中分裂，以便单个流变成一组流。一旦任何网络数据流分裂，就无法将这些流重新连接回单个数据流数据流，而不存在网络拓扑中出现环路的风险。要了解这与网络建模有何关系，请考虑瀑布模型中生成树和路由操作之间的差异。

The waterfall model shows how data flow splits in a network at every network device so that a single stream becomes a set of streams. Once any network data stream has split, there is no way to rejoin the streams back into a single stream of data without risking loops in the network topology. To understand how this relates to modeling a network, consider the difference between spanning tree and routing operation in a waterfall model.

生成树构建一棵大树，由每个源和目标对共享。这棵树意味着所有交通都必须“跃上瀑布”到达头端，然后才能跟随水流回到真正的目的地。例如，沿着流 W 流向流 Z 末尾的数据包必须沿着流到达其 A 处的源，然后沿着流 Z 的路径。

Spanning tree builds one large tree which is shared by every source and destination pair. This single tree means that all traffic must “leap up the waterfall” to the head end, where it can then follow the flow of water back toward its real destination. For instance, a packet flowing along Stream W toward the end of Stream Z must follow the stream to its source at A, and then follow the path for Stream Z.

对于路由控制平面，每个边缘设备都会构建自己的树，因此每个设备都位于瀑布或生成树的头端。每条交通流都沿着自己的一组溪流流下，而不是“跃上”瀑布到达顶部，然后流回瀑布。在此给出的示例中，如果流量从流 W 的入口点流向流 Z 的端点，它将遵循源自路由器 C 的不同树。这就是路由协议比生成树更高效的原因；流量不需要“跳跃”树来到达目的地。

For a routed control plane, each edge device builds its own tree, so each device is at the head end of a waterfall, or a spanning tree. Rather than “leaping up” the waterfall to reach the top, and then flowing back down, each traffic stream follows its own set of streams down. In the example given here, if traffic were flowing from an entrance point at Stream W to an endpoint at Stream Z, it would follow a different tree originating at Router C. This is why a routing protocol is more efficient than spanning tree; traffic doesn’t need to “leap up” the tree to reach its destination.

网络中的地方

Places in the Network

PIN 是一种沿着功能链路而不是拓扑链路划分网络的方法。图 6.8说明了网络的 PIN 视图。

PINs are a way to divide a network along functional, rather than topological links. Figure 6.8 illustrates a PINs view of a network.

图 6.8 网络中的位置

Figure 6.8 Places in the Network

网络的每个功能部分被分为不同的组件，例如两个数据中心和两个园区，并且它们通过互连点（图中标记为IC）进行连接。以这种方式分割网络可以强调网络每个部分的功能。每个部分可以使用不同的设计范例，以匹配所设计的 PIN 的特定用途。例如，大规模的中心辐射型拓扑可能在零售环境中占主导地位，而第一个数据中心可能使用传统的交换以太网拓扑进行设计，第二个数据中心则采用 Clos 拓扑。

Each functional section of the network is separated into a different component, such as the two data centers and the two campuses, and they are connected using interconnection points (marked IC in the illustration). Splitting the network up in this way emphasizes the function of each piece of the network. Different design paradigms can be used in each section, to match the specific purpose of the PIN being designed. For instance, a large-scale hub and spoke topology might dominate the retail environment, while the first data center might be designed using a traditional switched Ethernet topology, and the second as a Clos topology.

每个 PIN 的安全性、管理机制和其他方面也可以不同；数据中心 1 可能在数据中心内部具有开放的安全策略，并且对外部访问有严格的限制，而数据中心 2 可能没有入口策略，但每个服务器/应用程序的安全机制都很强大。每个 PIN 码对于网络中的其他 PIN 码都是完全不透明的。数据中心 1 只是数据中心 2 的流量接收器，WAN PIN 只是到达网络中其他 PIN 的传输机制。这是严格类型模块化的一种形式。

Security, management mechanisms, and other aspects of each PIN can also be different; Data Center 1 might have an open security policy within the data center itself, and strong restrictions on outside access, while Data Center 2 might have no entrance policies, but strong per server/application security mechanisms. Every PIN is completely opaque to every other PIN in the network. Data Center 1 is simply a traffic sink for Data Center 2, and the WAN PIN is simply a transport mechanism to reach the other PINs in the network. This is a form of strictly typed modularity.

连接这些不同的 PIN 是一系列互连，如图6.8中的浅灰色圆圈所示。某些 PIN 可能会直接相互连接以及连接到广域网 (WAN) 或核心 PIN，如 DMZ 所示和数据中心 2 个 PIN。其他人可能只连接到核心。每个 PIN 的内部结构与其他 PIN 完全不同。例如，数据中心 1 可能具有核心层、分布层和接入层，而数据中心 2 可能仅具有核心层和聚合层。这些层完全独立于整体网络设计。

Connecting these different PINs is a series of interconnects, shown as light gray circles in Figure 6.8. Some PINs might connect directly to each other as well as to the Wide Area Network (WAN), or Core PIN, as illustrated with the DMZ and Data Center 2 PINs. Others might connect only to the Core. Each PIN’s internal structure is completely different from every other PIN’s. For instance, Data Center 1 might have core, distribution, and access layers, and Data Center 2 might have only a core and aggregation layers. These layers are completely independent of the overall network design.

PIN 对于从操作角度理解网络设计非常有用，因为它们提供了基于每个 PIN 业务用途的强大功能视图。这使得每个业务问题都可以独立处理，这通常可以澄清所涉及的问题。供应商销售人员倾向于几乎完全在 PIN 内工作，因为它有助于将解决方案缩小到特定环境，从而有助于推动需求。

PINs are useful for understanding a network design from an operational perspective, because they provide a strong functional view based on business use for each PIN. This allows each business problem to be approached independently, which can often clarify the problems involved. Vendor sales folks tend to work within PINs almost exclusively, because it helps to narrow the solution to a particular environment, helping to drive requirements.

作为一种模型，PIN 在一个特定方面失败了——它们将网络架构集中在自下而上的功能视图上。这确实允许网络更紧密地模仿所需的功能；它将整体架构推到了视线之外。这可能会导致系统架构“有机增长”，而不是产生经过深思熟虑的总体规划和架构。

As a model, PINs fail in one particular regard—they focus the network architecture on a bottom up view of functionality. This does allow the network to more closely mimic the functionality required; it pushes the overall architecture out of sight. This can result in a systemic architecture that “just grows organically,” rather than producing a well thought out overall plan and architecture.

笔记

Note

PIN 的其他几个积极和消极方面与本章前面讨论的可互换模块的积极和消极方面类似。

Several other positive and negative aspects of PINs are similar to the positive and negative aspects of interchangeable modules discussed earlier in this chapter.

分层的

Hierarchical

基于无标度网络的分层网络模型与网络本身一样古老。本质上，分层设计就是采用模块化规则，将其与流量的瀑布模型相结合，最后将两者与信息隐藏的聚合相结合，并构建一套通常适用于大约任何网络设计项目。图 6.9说明了基本的分层设计。

Hierarchical network models, grounded in scale free networks, are as old as networks themselves. Hierarchical design is, in essence, taking the rules of modularity, combining them with a waterfall model of traffic flow, and finally combining these two with aggregation for information hiding, and building a set of “rules of thumb” that generally work for just about any network design project. Figure 6.9 illustrates a basic hierarchical design.

图 6.9 基本的分层设计

Figure 6.9 A Basic Hierarchical Design

笔记

Note

Cisco Press 的书籍《最佳路由设计》详细介绍了分层网络设计。²现在，许多网络都是两层设计，而不是三层，并采用“层内层”的方式进行扩展。

Hierarchical network design is covered in detail in the Cisco Press book, Optimal Routing Design.² Many networks are now two layer designs, rather than three, with “layers within layers” to build out to scale.

2 . Russ White、Alvaro Retana 和 Don Slice，《最佳路由设计》（Cisco Press，2005 年）。

2. Russ White, Alvaro Retana, and Don Slice, Optimal Routing Design (Cisco Press, 2005).

从网络复杂性的角度考虑分层网络设计的一些“经验法则”。

Consider some of the “rules of thumb” for hierarchical network design in network complexity terms.

集中分层功能

分层设计中的三个层（访问层、分发层和核心层）中的每一层都应专注于一小组功能或目的。例如，网络核心应该专注于在分布层模块之间转发流量，而不是执行策略，甚至连接到外部网络。例如，互联网和外部合作伙伴网络不应连接到网络核心，而应连接到与其他接入层模块并行的接入层模块。

Each of the three layers in the hierarchical design—access, distribution, and core—should be focused on a small set of functions or purposes. For instance, the network core should be focused on forwarding traffic between the distribution layer modules, rather than implementation of policy, or even connectivity to outside networks. For instance, the Internet and outside partner networks should not be connected to the network core, but rather to an access layer module parallel to the other access layer modules.

将功能集中在每一层内有助于通过控制网络模块内部和之间的交互表面来管理复杂性。例如，通过专注于分布层模块之间的转发，可以大大简化网络核心设备的配置——准入策略和每个用户或设备的安全性可以从这些设备中剔除，因为这些功能是在网络中的其他地方处理的。限制任何特定功能的位置还可以简化设备选择，并允许层内的模块更加相似。例如，如果长距离暗光纤接口的高速转发是一致推送到网络核心设备的功能，则可以选择分布层设备，而无需考虑需要支持的接口类型这个能力。

Focusing functionality within each layer helps manage complexity by controlling the interaction surfaces within and between the network modules. By focusing on forwarding between distribution layer modules, for instance, the configuration of the network core devices can be greatly simplified—admittance policy and per user or device security can be left off these devices, as those functions are handled someplace else in the network. Restricting the location of any particular function also simplifies equipment choices, and allows the modules within the layer to be more similar. As an example, if high-speed forwarding over a long distance dark fiber interface is a function consistently pushed to the network core devices, then distribution layer devices can be chosen without reference to the types of interfaces required to support this capability.

政策重点

分层设计模式还在网络拓扑中提供了方便的“瓶颈”。在拓扑的边缘，沿着用户连接点到接入层，并且在每层之间，有较少数量的连接将拓扑拉在一起。这些地方的连接量更多地受到网络设计的限制，是以更集中的方式实施政策的完美场所。

The hierarchical design pattern also provides convenient “choke points” in the network topology. At the edge of the topology, along the user connection point to the access layer, and between each layer, there are a smaller number of connections that pull the topology together. These places, where the amount of connectivity is more limited by the design of the network, are perfect places to implement policy in a more centralized way.

作为说明，将聚合视为一种策略。在分层设计中的层之间的链接上配置聚合是有意义的，如下所示：

By way of illustration, consider aggregation as a policy. It makes sense to configure aggregation on the links between layers in a hierarchical design, as this:

• 提供每个模块内的完整路由信息。

• Provides full routing information within each module.

• 提供一小组位置来查找每个模块内的聚合。

• Provides a small set of places to look for aggregation within each module.

• 提供一小组位置来查找模块之间的次优流量分配。

• Provides a small set of places to look for suboptimal traffic distribution between the modules.

• 在模块之间的控制平面状态之间提供分隔点。

• Provides a separation point between the control plane state between modules.

在拓扑中提供“阻塞点”允许网络模块之间的受控交互表面，在这些点上减少模块之间承载的控制平面状态，进而提供一种控制控制平面必须的速度的方法管理网络变更。

Providing “choke points,” in the topology allows for controlled interaction surfaces between the network modules, points at which to reduce the control plane state being carried between modules, and, in turn, a way to control the speed at which the control plane must manage network changes.

笔记

Note

围绕聚合的权衡将在本章前面的信息隐藏部分和第 5 章“设计复杂性”中讨论。围绕策略布局的权衡将在第 5 章“设计复杂性”中讨论。

The tradeoffs around aggregation are discussed in the Information Hiding section earlier in this chapter and Chapter 5, “Design Complexity.” The tradeoffs around policy placement are discussed in Chapter 5, “Design Complexity.”

统一建模语言

UML

统一建模语言（UML）在这里可能看起来不合适，因为它是一个建模工具，而不是一个模型或系统。然而，网络工程界往往“凭感觉行事”，而不是有意思考网络与应用程序和策略交互的方式。以流程为中心的建模语言对于理解应用程序的工作细节非常有帮助，然后可以将其直接映射到网络中的策略和数据包流。在大规模数据中心和云部署中尤其如此。尽管大规模云和数据中心部署被设计为“与应用程序无关”，但在了解数据包流以及策略实施位置时，通常需要对数据中心结构上运行的应用程序进行故障排除，

The Unified Modeling Language (UML) might seem out of place here, because it’s a modeling tool, rather than a model or a system. However, the network engineering world tends to “fly by the seat of its pants,” rather than intentionally think through the way the network interacts with applications and policies. A process focused modeling language can be really helpful in understanding how an application works in some detail, which can then be directly mapped to policies and packet flows in the network. This is particularly true in large-scale data center and cloud deployments. Although large-scale cloud and data center deployments are designed to be “application agnostic,” there often comes a point in troubleshooting an application running on a data center fabric when understanding the packet flow, and where policies are put in place, is really helpful in determining where the problem lies.

图 6.10说明了 Web 应用程序的 UML 模型。

Figure 6.10 illustrates a UML model for a web application.

图 6.10 使用 UML 建模的简单 Web 应用程序示例

Figure 6.10 Example of a Simple Web Application Modeled Using UML

虽然图 6.10所示的图重点关注组成 Web 应用程序的不同进程之间的交互，但应用程序之间的交互都是通过网络发生的，因此，每次交互都代表必须规划的数据包流。检查此图，网络工程师可能会提出以下问题：

While the diagram shown in Figure 6.10 focuses on the interaction between the different processes which make up the web application, the interactions between the applications all happen across the network—as such, each interaction represents a flow of packets that must be planned for. Examining this diagram, a network engineering might ask questions like:

• 每个进程之间使用什么类型的连接？如果是第 2 层连接，则必须将进程放入单个第 2 层域中，或者必须以其他方式处理流量，以便进程相信它们是通过第 2 层链路进行连接的。

• What type of connectivity is used between each of these processes? If it’s Layer 2 connectivity, then the processes must be placed into a single Layer 2 domain, or the traffic must otherwise be handled so the processes believe they are connected over a Layer 2 link.

• 每个连接使用什么协议？例如，TCP 可以用在超文本标记语言（HTML）渲染组件和会话控制之间，但用户数据报协议（UDP）可以用在会话控制和逻辑组件之间。这些信息将在确定需要部署的位置和类型的服务质量方面发挥重要作用。

• What protocol is used at each connection? For instance, TCP might be used between the Hypertext Markup Language (HTML) Render component and session control, but User Datagram Protocol (UDP) might be used between session control and the logic components. This information will play a large role in determining where and what types of quality of service need to be deployed.

• 政策在哪里以及如何实施？图中只标注了两个策略（与大多数现实世界的应用程序部署相比，这相当简单）——这些策略是由网络、应用程序还是两者实现的？如何实施？

• Where and how is policy implemented? There are only two policies noted on the diagram (this is rather simple compared to most real-world application deployments)—are these policies implemented by the network, the application, or both? How are they to be implemented?

除了这些问题之外，对每个流量的大小进行可靠的估计对于构建应用程序使用网络资源的方式非常有帮助。如果您有每个应用程序的此类信息，那么它可以将这些信息汇总为网络上流量的整体情况，但即使在单个应用程序级别，了解这些类型的流量也很有用。

Beyond these questions, getting a solid estimate of the sizes of each traffic flow could be very helpful in building a picture of the way the application uses network resources. If you have this type of information for each application, it is possible to roll this information up into an overall picture of what the traffic should look like on the network—but even at the individual application level, understanding these sorts of traffic flows can be useful.

最后的想法

A Final Thought

从复杂性的角度来看，网络设计人员和架构师面临着非常不确定的环境。虽然这些工具“已经存在”，但它们在管理复杂性方面通常没有被广泛理解，而且它们也不总是得到很好的开发。相反，大多数网络设计人员都根据“经验法则”进行工作，基于“凭感觉行事”，基于多年的无效经验。

Network designers and architects face very uncertain environments from a complexity perspective. While the tools are “out there,” they aren’t often widely understood in terms of managing complexity, nor are they always well developed. Instead, most network designers work from “rules of thumb,” based on “seat of the pants flying,” based on long years of experience with what doesn’t work.

通过从复杂性的角度检查“经验法则”，您可以开始在它们背后提出一些推理——它们为什么有效、在哪里有效以及何时无效。从复杂性的角度审视网络设计问题有助于解开一些谜团，并通过更深入地了解真正的权衡以及技术和模型如何真正发挥作用，为更好的网络设计指明道路。

By examining the “rules of thumb” from a complexity perspective, you can begin to put some reasoning behind them—why they work, where they work, and when they won’t work. Examining the problems of network design from a complexity perspective helps to untangle some of the mysteries, and to point the way toward better network design through a deeper understanding of what the tradeoffs really are, and how the techniques and models really work.

第 7 章协议复杂性

Chapter 7. Protocol Complexity

谁关心协议的复杂性？这是一个为那些热衷于基数树和深入数学的极客们保留的话题，对吧？

Who cares about protocol complexity? It’s a topic reserved for the geeks with their heads in radix trees and deep math, right?

错了。

Wrong.

网络上部署的协议，无论是用于控制平面还是通过网络传输数据，实际上本身就是系统（通常是复杂的系统），它们沿着所讨论的相同类型的交互表面与网络中的其他系统进行交互在网络和设计层面。由于这些设计表面，“深入协议”工作的网络工程师需要了解网络设计，就像网络架构人员需要了解协议设计一样。

The protocols deployed on a network, whether used in the control plane or to carry data through the network, are actually systems in their own right—often complex systems—that interact with the other systems in the network along the same sorts of interaction surfaces discussed at the network and design levels. Because of these design surfaces, network engineers working “down in the protocols” need to know network design just as much as network architecture folks need to know protocol design.

本章旨在将复杂性权衡讨论带入协议设计领域，以便网络工程师能够理解为什么一种协议在一种情况下可能效果更好，而其他协议在另一种情况下效果更好。网络工程领域有两种相反的趋势需要警惕：

This chapter aims to bring the complexity tradeoffs discussion to the world of protocol design, so network engineers can understand why one protocol might work better in one situation, and others in another. There are two opposite trends in the network engineering world to guard against:

• 尝试用单一协议解决所有问题；这在 BGP 驱动器中最为明显（截至发稿时）。虽然针对所有事物的单一控制平面（以及一个环来统治所有事物）可以极大地简化网络设计人员的工作，但它也可能会推动次优设计。

• Trying to solve every problem with a single protocol; this is most pronounced in the drive with the BGP (at presstime). While a single control plane for everything (and one ring to rule them all) can greatly simplify the job of the network designer, it can also push suboptimal designs.

• 用自己的协议套件解决每一个问题。

• Solving every problem with its own protocol suite.

这里的权衡并不总是显而易见的，但显然需要在为每个问题部署单点解决方案和为所有问题部署单一解决方案之间取得平衡。这就是网络设计中分层的地方（虚拟化拓扑，以便不同的控制和数据平面看到不同的网络视图）可能非常有用。

The tradeoffs here aren’t always obvious, but there clearly needs to be a balance between deploying a point solution for every problem and deploying a single solution for all problems. This is where layering in a network design (virtualizing topologies so different control and data planes see different views of the network) can be very useful.

本章首先考虑灵活性和复杂性之间的权衡，使用 OSPF 与 IS-IS 作为“经典”示例，说明不同的复杂性权衡如何导致完全不同的协议，每个协议都有自己的一组权衡。第二部分考虑分层与复杂性，特别是根据 John Day 的网络传输迭代模型。第三部分使用两个示例来考虑协议复杂性与设计复杂性；第一个是回归链路状态协议中的微循环和快速收敛概念，第二个考虑了“早年”对 EIGRP 的看法与现实的对比，以及多年来为控制网络复杂性而对协议进行的更改设计。

This chapter begins with a section considering the tradeoff between flexibility and complexity, using OSPF versus IS-IS as the “classic” example of how different complexity tradeoffs lead to completely different protocols, each with their own set of tradeoffs. The second section considers layering versus complexity, specifically in light of John Day’s iterative model of network transport. The third section considers protocol complexity against design complexity using two examples; the first is a return to the concepts of microloops and fast convergence in link state protocols, and the second considers the perception of EIGRP in the “early years” against the reality, and changes made to the protocol over the years to contain complexity in network design.

灵活性与复杂性：OSPF 与 IS-IS

Flexibility versus Complexity: OSPF versus IS-IS

也许协议设计领域复杂性与灵活性的经典例子就是 OSPF 和 IS-IS 最初的对比设计。需要了解一点历史才能理解与这两个协议相关的讨论和决策的原始背景。

Perhaps the classic example of complexity versus flexibility in the world of protocol design is the original contrasting designs of OSPF and IS-IS. A little bit of history is required to understand the original context of the discussion and decisions that were made in relation to the two protocols.

笔记

Note

IS-IS 是一种广泛部署在大型服务提供商网络中的链路状态协议。该协议的标准规范包含在 ISO 10589 中。¹

IS-IS is a link state protocol widely deployed in large-scale service provider networks. The standards specification for the protocol is contained in the ISO 10589.¹

1 . 有关 IS-IS 的更多信息，请参阅 Russ White 和 Alvaro Retana，IS-IS：IP 网络中的部署，第 1 版。（波士顿：艾迪生韦斯利，2003 年）。

1. For more information on IS-IS, see Russ White and Alvaro Retana, IS-IS: Deployment in IP Networks, 1st edition. (Boston: Addison-Wesley, 2003).

IS-IS 最初设计用于支持开放系统互连（OSI）协议栈（最初的七层堆栈），该协议栈是围绕终端系统（ES）和中间系统（IS）设计的。每个 ES 都根据设备的第 2 层地址分配一个地址，然后由连接的 IS 在整个网络中进行通告 — 本质上，主机路由是基于 OSI 的网络的正常操作模式，仅在泛洪时进行聚合域边界。所涉及的实际系统是重量较重的计算机（当时），因此具有良好的处理性能和可用内存，因此 IS-IS 的原始设计者更关心设计一种灵活且可扩展至新地址类型的协议（以防 OSI 网络协议采用新型终端主机进行路由，或层2 寻址更改等），以及以最少数量的数据包类型处理 1 级和 2 级洪泛域的能力。

IS-IS was originally designed to support the Open Systems Interconnect (OSI) protocol stack (the original seven-layer stack), which was designed around End Systems (ES) and Intermediate Systems (IS). Each ES was assigned an address based on the Layer 2 address of the device that was then advertised by the connected IS throughout the network—essentially host routing was the normal mode of operation for OSI-based networks, with aggregation only taking place at the flooding domain boundary. The actual systems involved were heavier weight computers (for that time), and hence had a good deal of processing performance and available memory, so the original designers of IS-IS were more concerned about designing a protocol that was flexible and extensible to new address types (in case the OSI networking protocols ever took on a new type of end host to route for, or Layer 2 addressing changed, etc.), as well as the ability to handle level 1 and level 2 flooding domains with a minimal number of packet types.

这种总体前景催生了一个专注于在 TLV 中携带信息的协议；要添加新信息，可以轻松地将携带该信息的新 TLV 添加到协议中。同时，由单个设备发起的所有信息都承载在单个相应的链路状态数据包 (LSP) 中，如果数据包的大小大于网络的最大传输单元 (MTU)，则该数据包可能会被分段。

This general outlook led to a protocol that was focused on carrying information in TLVs; to add new information, a new TLV to carry the information could easily be added to the protocol. At the same time, all the information originated by a single device was carried in a single, corresponding, Link State Packet (LSP), which could be fragmented if the size of packet became larger than the Maximum Transmission Unit (MTU) of the network.

另一方面，OSPF 的设计目标略有不同。在设计 OSPF 期间，路由器（尤其是路由器）拥有今天被认为性能非常低的处理器。作为特殊用途的设备，路由器根本不具备任何给定 ES 可能拥有的处理性能或内存，因此，虽然 IS-IS 依靠相对高性能的设备来处理转发，但 OSPF 是围绕相对较低的概念开发的。 - 处理转发的性能设备。

OSPF, on the other hand, was designed with a slightly different set of goals in mind. In the time during which OSPF was designed, routers (in particular) had what would today be considered very low performance processors. As special purpose devices, routers simply were not equipped with the processing performance or the memory that any given ES might have—so while IS-IS counted on a relatively high-performance device handling forwarding, OSPF was developed around the concept of a relatively low-performance device handling forwarding.

为了实现这些目标，OSPF 设计中首先考虑了三个具体要点：

To meet these goals, three specific points came to the forefront in OSPF’s design:

• 固定长度编码。OSPF 不使用 TLV 来编码和携带信息，而是专注于使用固定长度编码，这在线路（可以省去 TLV 标头）和处理（可以使用固定代码块来处理特定数据包）方面需要较少的资源格式，而不是动态读取单个流中携带的各种 TLV 并对其做出反应所需的更长、更复杂的代码块）。

• Fixed length encoding. Rather than using TLVs to encode and carry information, OSPF focused on using fixed length encoding, which requires less resources both on the wire (the TLV header can be dispensed with) and in processing (fixed blocks of code can be used to process specific packet formats, rather than the longer, more complex blocks of code required to dynamically read and react to various TLVs carried within a single stream).

• 修复了较小尺寸的数据包类型。OSPF 专注于较短的数据包（理论上）适合网络上的单个 MTU 数据包，而不是每个路由器都生成一个碎片化的单个“数据包”，并解决同步这些碎片带来的众多麻烦。这使得泛洪和同步变得更加简单。

• Fixed packet types of a smaller size. Rather than each router originating a single “packet” that is fragmented, and the numerous headaches around synchronizing these fragments, OSPF focused on shorter packets that would (in theory) fit in a single MTU packet on the network. This made flooding and synchronization simpler.

• 聚合可达性。在 IP 中，第 3 层寻址与第 2 层寻址无关，这意味着（实际上）可达性是从第一跳开始聚合的。IP 路由器通告子网，子网是一组主机的集合，而不是每个单独的主机。这意味着在设计的初始阶段，OSPF 需要以比 IS-IS 更以 IP 为中心的方式来处理多路访问链路和聚合。

• Aggregated reachability. In IP, Layer 3 addressing is untied from Layer 2 addressing, which means (in practice) that reachability is aggregated from the first hop. IP routers advertise subnets, which are an aggregate of a set of hosts, rather than each individual host. This means OSPF needed to handle multiaccess links and aggregation in a way that was more IP centric than IS-IS in the initial phases of the design.

由于这些目标，OSPF 被设计为具有多种不同的 LSA 类型，每种类型都有固定的格式，并且每种类型都携带不同类型的信息。图 7.1说明了其中的差异。

As a result of these goals, OSPF was designed with a number of different LSA types, each one having a fixed format, and each one carrying a different type of information. Figure 7.1 illustrates the difference.

图 7.1 OSPF 与 IS-IS

Figure 7.1 OSPF versus IS-IS

图 7.1所示的网络仅包含两个路由器，每个路由器都连接到末节链路，并且都连接到公共广播（多点）链路。OSPF 和 IS-IS 都会选举一个伪节点（尽管它称为指定路由器OSPF 中的 [DR]）代表广播链路，以减少流经该链路的邻接数量和路由协议流量；这由路由器 B 表示，它较小且呈灰色。将每个 IS-IS TLV 视为单独的数据包（或单个 LSP 的片段），每个协议生成的两组数据包并排设置。如果您了解链路状态路由的概念，那么结果就不足为奇了——两种协议中必须以某种方式携带相同的信息。主要区别在于两种协议如何编码信息。IS-IS 将信息编码在片段内的 TLV 中，而 OSPF 将信息编码在不同类型的数据包中。如果IS-IS中的每个TLV代表OSPF中的单个LSA类型，则相关性几乎是一对一的。

The network illustrated in Figure 7.1 consists of only two routers, each connected to a stub link, and both connected to a common broadcast (multipoint) link. Both OSPF and IS-IS elect a pseudonode (though it’s called a Designated Router [DR] in OSPF) to represent the broadcast link to reduce the number of adjacencies and routing protocol traffic flowing across the link; this is represented by Router B, which is smaller and gray. Treating each IS-IS TLV as a separate packet (or fragment of a single LSP), the two sets of packets generated by each protocol are set side-by-side. If you understand the concept of link state routing, the result isn’t very surprising—the same information must somehow be carried in both protocols. The main difference is in how the two protocols encode the information. IS-IS encodes the information in TLVs within fragments, while OSPF encodes the information in different types of packets. If each TLV in IS-IS represents a single LSA type in OSPF, the correlation is almost one-to-one.

OSPF 权衡了编码和传输的复杂性与固定长度数据包的关系。这就引出了一个问题：使用 TLV 给 IS-IS 带来什么优势？TLV的代价是数据包尺寸更大，处理更复杂；IS-IS 如何通过这种方式增加数据包格式的复杂性来对抗 OSPF？

OSPF traded off complexity in coding and transport against fixed length packets. This leads to the question—what advantage does using TLVs give IS-IS? The cost of the TLV is a larger packet size and more complex processing; how does IS-IS gain against OSPF for increasing the packet formatting complexity in this way?

最明显的答案是能够快速、轻松地向 IS-IS 添加新型信息。例如，当 IPv6 开发出来时，这两种链路状态协议都需要支持新的 IP 格式。IS-IS 只是添加了一组新的 TLV，而 OSPF 则需要一个全新的协议——OSPFv3。固定长度格式的缺点是显而易见的。一般来说，IS-IS 增加了TLV 类型，而OSPF 增加了报文类型。

The most obvious answer is in the ability to quickly and easily add new types of information to IS-IS. For instance, when IPv6 was developed, both of these link state protocols needed to support the new IP format. IS-IS simply added a new set of TLVs, while OSPF required an entirely new protocol—OSPFv3. The downside of fixed length formats is immediately obvious. In general, IS-IS proliferates TLV types, while OSPF proliferates packet types.

笔记

Note

这是为了说明特定点而进行了一些简化；TLV 格式并不是创建 OSPFv3 而不是简单地将 IPv6 添加到 OSPFv2 的唯一原因。例如，OSPFv2 跨 IPv4 层建立邻接关系并承载 LSA，因此 OPSFv2 中的下一跳是 IPv4 地址，这在尝试部署 IPv6 时会引起一些麻烦，因为 IPv6 自然应该使用 IPv6 下一跳。与直接与第 2 层对等的 IS-IS 相比，OSPF 与底层第 3 层传输的联系更为紧密。

This is a bit simplified to illustrate a specific point; the TLV formatting is not the only reason why OSPFv3 was created rather than simply adding IPv6 to OSPFv2. For instance, OSPFv2 builds adjacencies, and carries LSAs, across layer IPv4, so the next hops in OPSFv2 are IPv4 addresses—which would cause some headaches when trying to deploy IPv6, which should naturally use IPv6 next hops. OSPF is more intimately tied to the underlying Layer 3 transport than IS-IS, which peers directly at Layer 2.

OSPF 好还是 IS-IS 好？这个问题没有明确的答案——很大程度上取决于协议的用途、网络运营商的技能以及个人的实现。在现实世界中，多年前根据每个协议设计的不同环境所做的权衡已经被事件所克服；任何关于这两个协议的辩论都必须达到详细程度，才能真正找到值得一提的差异，这一点很明显，细节并不重要。

Is OSPF better, or IS-IS? There’s no clear cut answer to this question—a lot depends on the use to which the protocol is being put, the skills of the network operators, and the individual implementations. In the real world, the tradeoffs made years ago based on the different environments each protocol was designed for have been overcome by events; that the details hardly matter is apparent in the level of detail any debate about the two protocols must reach to actually find a difference worth mentioning.

分层与协议复杂性

Layering versus Protocol Complexity

图 7.2展示了两个协议栈；哪一个更简单？

Figure 7.2 illustrates two protocol stacks; which one is simpler?

图 7.2 协议栈

Figure 7.2 Protocol Stacks

如果您像大多数网络工程师一样，您可能对此图有以下两种反应之一：

If you’re like most network engineers, you’re likely to have one of two reactions to this illustration:

• XYZ 协议，因为堆栈中只有一种协议，并且一种协议总是比许多交互协议更简单。

• The XYZ protocol, because there’s only one protocol in the stack—and one protocol is always simpler than many protocols interacting.

• XYZ 协议栈，因为分层很好。传输协议（例如 TCP 和 IP）不也是这样分层的吗？

• The XYZ protocol stack, because layering is good. Aren’t transport protocols (such as TCP and IP) layered like this?

要么您认为分层是好的，因为每个网络工程师在第一次使用控制台时都已将它们深深地铭记在心，要么分层是一种必要的罪恶，让您在坐下来参加认证测试时很难记住分层或大学考试。是哪一个？

Either you believe that layers are good because they’ve been drilled into the head of every network engineer because of their first console experience, or layers are a necessary evil that make it hard to remember when you’re sitting down to take a certification test or college exam. Which is it?

两者都不是。

It’s neither.

哪一个更复杂实际上取决于几个因素，包括：

Which one is more complex actually depends on several factors, including:

• 协议栈的总功能是什么。改变插图的背景会改变对问题的看法；如果有人说，“XYZ 协议栈是当前传输栈中 IP 组件的替代品，将该栈中的层从四层增加到六层”，你会立即认为“太复杂了”。如果有人说，“XYZ 协议用单个隧道协议取代了当前在网络中运行的三层隧道”，您可能会立即大喊“万岁！”

• What the total functionality of the stack of protocols is. Changing the context of the illustration changes the perception of the problem; if someone said, “the XYZ protocol stack is a replacement for the IP component in the current transport stack, increasing the layers in that stack from four to six,” you’d immediately think “too complex.” If someone said, “the XYZ protocol replaces three layers of tunnels currently running in the network with a single tunneled protocol,” you’d probably immediately yell, “hurrah!”

• 堆栈中协议之间的交互界面定义得如何。具有明确定义的层的协议栈可以形成明确定义的交互界面。明确定义的交互表面仍然是有漏洞的抽象，但更好的定义往往会减少泄漏。明确定义的交互表面仍然是增加复杂性的交互表面，但明确定义的深度和范围都有助于减少跨层抽象泄漏，并且更容易理解交互以进行修改和故障排除。

• How well defined the interaction surfaces are between the protocols in the stack. Protocol stacks with well-defined layers make for well-defined interaction surfaces. Well-defined interaction surfaces are still leaky abstractions, but better definition tends to reduce the leaks. Well-defined interactions surfaces are still interaction surfaces that add complexity, but well-defined depth and scope both contribute to less cross-layer abstraction leaks, and make it easier to understand the interactions for modifications and troubleshooting.

因此，从复杂性的角度来看，分层可能是一件好事，也可能是一件坏事。这完全取决于分层如何影响以下方面：

Layering, then, can either be a good thing or a bad thing from a complexity standpoint. It all depends on how the layering impacts the following:

•状态：添加一层实际上是否以某种有意义的方式划分状态？或者附加的层只是简单地分散通常跨越分层边界“一起作用”的状态，从而使交互表面更加复杂？

• State: Does the addition of a layer actually divide state in some meaningful way? Or are the additional layers simply spreading out state that often “acts together” across the layering boundary, making the interaction surface more complex?

•速度：附加层是否会降低一层必须运行的速度，或者将两个具有完全不同“运行节奏”的进程分开？

• Speed: Does the additional layer reduce the speed at which one layer must operate, or separate two processes with completely different “operation tempos”?

•表面：附加层是否会产生一组干净的交互点并最大限度地减少泄漏抽象？是否只有少数几个地方会产生意想不到的后果，还是很多地方？

• Surface: Does the additional layer produce a set of clean interaction points and minimize leaky abstractions? Are there just a few places where unintended consequences can come into play, or many?

笔记

Note

意外后果的概念与辅助性概念密切相关，第 10 章“可编程网络复杂性”对此进行了更详细的介绍。

The concept of unintended consequences is closely tied to the concept of subsidiarity, which is covered in more detail in Chapter 10, “Programmable Network Complexity.”

网络工程师处理上述有关协议栈分层问题的方法之一是根据模型构建协议栈。两种常见的模型是四层DoD 模型和七层OSI 模型。虽然大多数网络工程师至少熟悉其中一种模型，但可能需要快速回顾一下。

One of the ways network engineers have dealt with the questions raised above about layering in protocol stacks is by building protocol stacks according to models. Two common models are the four-layer DoD model, and the seven-layer OSI model. While most network engineers are familiar with at least one of these models, a quick review might be in order.

七层模型

The Seven-Layer Model

事实上，任何参加过网络课程或学习过网络工程认证的人都熟悉使用七层模型来描述网络的工作方式。无连接网络协议 (CLNP) 和路由协议 IS-IS 是由 ISO 设计的，以满足七层模型中给出的要求。这些协议仍在广泛使用，特别是 IS-IS，它已被修改以支持 IP 网络中的路由。图 7.3说明了七层模型。

Virtually anyone who has ever been through a networking class or studied for a network engineering certification is familiar with using the seven-layer model to describe the way network’s work. Connectionless Networking Protocol (CLNP) and a routing protocol, IS-IS, were designed by the ISO to meet the requirements given within the seven-layer model. These protocols are still in wide use, particularly IS-IS, which has been modified to support routing in IP networks. Figure 7.3 illustrates the seven-layer model.

图 7.3 七层模型

Figure 7.3 Seven-Layer Model

每对层在模型中垂直移动，并通过 API 进行交互。因此，要连接到特定的物理端口，数据链路层的一段代码将连接到该端口的套接字。这使得各层之间的交互能够被抽象和标准化。一块网络层软件不需要知道如何处理各种物理接口，只需要知道如何将数据传送到同一系统上的数据链路层软件即可。

Each pair of layers, moving vertically through the model, interacts through an API. So to connect to a particular physical port, a piece of code at the data link layer would connect to the socket for that port. This allows the interaction between the various layers to be abstracted and standardized. A piece of software at the network layer doesn’t need to know how to deal with various sorts of physical interfaces, only how to get data to the data link layer software on the same system.

每层都有一组特定的功能要执行：

Each layer has a specific set of functions to perform:

• 物理层（第1 层）负责将0 和1 调制或串行化到物理链路上。每种链路类型都有不同的格式来发送 0 或 1；物理层负责将 0 和 1 转换为这些物理信号。

• The physical layer (Layer 1) is responsible for getting the 0s and 1s modulated, or serialized, onto the physical link. Each link type has a different format for signaling a 0 or 1; the physical layer is responsible for translating 0s and 1s into these physical signals.

• 数据链路层负责确保传输的信息发送到链路另一端的正确计算机。每个设备都有不同的数据链路（第 2 层）地址，可用于将流量发送到该特定设备。数据链路层假设信息流中的每个帧与同一流中的所有其他数据包分开，并且仅为通过单个物理链路连接的设备提供通信。

• The data link layer is responsible for making certain that transmitted information is sent to the right computer on the other side of the link. Each device has a different data link (Layer 2) address that can be used to send traffic to that specific device. The data link layer assumes that each frame within a flow of information is separate from all other packets within that same flow and only provides communication for devices that are connected through a single physical link.

• 网络层负责在未通过单个物理链路连接的系统之间传输数据。然后，网络层提供网络范围（或第 3 层）地址，而不是链路本地地址地址，并且还提供一些方法来发现到达这些目的地必须跨越的设备和链路集。

• The network layer is responsible for transporting data between systems that are not connected through a single physical link. The network layer, then, provides network-wide (or Layer 3) addresses, rather than link local addresses, and also provides some means for discovering the set of devices and links that must be crossed to reach these destinations.

• 传输层（第4 层）负责不同设备之间数据的透明传输。传输层协议可以是“可靠的”，这意味着传输层将重新传输在某个较低层丢失的数据，也可以是“不可靠的”，这意味着在较低层丢失的数据必须由某个较高层的应用程序重新传输。

• The transport layer (Layer 4) is responsible for the transparent transfer of data between different devices. Transport layer protocols can either be “reliable,” which means the transport layer will retransmit data lost at some lower layer, or “unreliable,” which means that data lost at lower layers must be retransmitted by some higher-layer application.

• 会话层（第5 层）并不真正传输数据，而是管理运行在两台不同计算机上的应用程序之间的连接。会话层确保数据的类型、数据的形式和数据流的可靠性都被暴露和考虑。

• The session layer (Layer 5) doesn’t really transport data, but manages the connections between applications running on two different computers. The session layer makes certain that the type of data, the form of the data, and the reliability of the data stream are all exposed and accounted for.

• 表示层（第6 层）以运行在两个设备上的应用程序可以理解和处理的方式格式化数据。加密、流量控制以及在应用程序和网络之间提供接口所需的任何其他数据操作都发生在这里。应用程序通过套接字与表示层交互。

• The presentation layer (Layer 6) formats data in a way that the application running on the two devices can understand and process. Encryption, flow control, and any other manipulation of data required to provide an interface between the application and the network happen here. Applications interact with the presentation layer through sockets.

• 应用层（第7 层）提供用户和应用程序之间的接口，应用程序又通过表示层与网络进行交互。

• The application layer (Layer 7) provides the interface between the user and the application, which in turn interacts with the network through the presentation layer.

模型中的每一层都提供其下面的层所携带的信息；例如，第 3 层提供第 2 层使用第 1 层封装和传输的位。这导致了以下观察：不仅可以在七层模型中以精确的术语描述各层之间的交互，而且在可以精确地描述多台计算机。第一设备上的物理层可以与第二设备上的物理层通信，第一设备上的数据链路层与第二设备上的数据链路层通信，等等。正如设备上两层之间的交互是通过套接字处理的一样，不同设备上并行层之间的交互也是通过网络协议处理的：

Each layer in the model provides the information the layer below it is carrying; for instance, Layer 3 provides the bits Layer 2 encapsulates and transmits using Layer 1. This leads to the following observation: not only can the interaction between the layers be described in precise terms within the seven-layer model, the interaction between parallel layers on multiple computers can be described precisely. The physical layer on the first device can be said to communicate with the physical layer on the second device, the data link layer on the first device with the data link layer on the second device, and so on. Just as interactions between two layers on a device are handled through sockets, interactions between parallel layers on different devices are handled through network protocols:

• 以太网描述了物理线路上的0 和1 信号、启动和停止数据帧的格式以及在连接到单条线路的所有设备中对单个设备进行寻址的方法。因此，以太网属于 OSI 模型中的第 1 层和第 2 层。

• Ethernet describes the signaling of 0s and 1s onto a physical piece of wire, a format for starting and stopping a frame of data, and a means of addressing a single device among all the devices connected to a single wire. Ethernet, then, falls within both Layer 1 and Layer 2 in the OSI model.

• IP 描述了将数据格式化为数据包的方式，以及通过多个第 2 层链路发送数据包以到达几跳之外的设备所需的寻址和其他方法。因此，IP 属于 OSI 模型的第 3 层。

• IP describes the formatting of data into packets, and the addressing and other means necessary to send packets across multiple Layer 2 links to reach a device that is several hops away. IP, then, falls within Layer 3 of the OSI model.

• TCP 描述会话设置和维护、数据重传以及与应用程序的交互。因此，TCP 属于 OSI 模型的传输层和会话层。

• TCP describes session setup and maintenance, data retransmission, and interaction with applications. TCP, then, falls within the transport and session layers of the OSI model.

图 7.3对此进行了说明，其中每个“逐层”交互都使用用于描述传输的信息块的名称进行标记。例如，段从一个设备上的传输层传输到另一设备，而数据包从一个设备上的网络层传输到另一设备。

This was illustrated in Figure 7.3, where each “layer-wise” interaction is labeled using the name used to describe the blocks of information transferred. For instance, segments are transferred from the transport layer on one device to another, while packets are transferred from the network layer on one device to another.

四层模型

The Four-Layer Model

工程师们很难将 IP 传输堆栈融入七层模型的原因之一是，它是在开发时考虑到了不同的网络通信模型。IP 传输堆栈并非采用七层模型，而是围绕四层模型进行设计，RFC 1122 中对此进行了粗略描述，并如图7.4所示。

One of the reasons why engineers have so much difficulty fitting the IP transport stack into the seven-layer model is that it was developed with a different model of network communications in mind. Instead of a seven-layer model, the IP transport stack was designed around a four-layer model, roughly described in RFC 1122 and illustrated in Figure 7.4.

图 7.4 四层模型

Figure 7.4 Four-Layer Model

在此模型中：

In this model:

• 链路层大致负责与OSI 模型中的物理和数据链路层相同的功能——控制物理链路的使用、链路本地寻址以及在单个物理链路上将帧作为单个位传送。

• The link layer is roughly responsible for the same functions as the physical and data link layers in the OSI model—controlling the use of physical links, link-local addressing, and carrying frames as individual bits across an individual physical link.

• 互联网层大致负责与 OSI 模型中的网络层相同的任务 — 提供跨多个物理链路的寻址和可达性，并提供单一数据包格式和接口，而不管实际的物理链路类型如何。

• The Internet layer is roughly responsible for the same things as the Network layer in the OSI model—providing addressing and reachability across multiple physical links and providing a single packet format and interface regardless of the actual physical link type.

• 传输层负责在通信设备之间建立和维护会话，并为数据流或数据块提供通用的透明数据传输机制。流量控制和可靠传输也可以在这一层中实现，就像 TCP 的情况一样。

• The transport layer is responsible for building and maintaining sessions between communicating devices and providing a common transparent data transmission mechanism for streams or blocks of data. Flow control and reliable transport may also be implemented in this layer, as in the case of TCP.

• 应用层是用户与网络资源或使用数据并向连接到网络的其他设备提供数据的特定应用程序之间的接口。

• The application layer is the interface between the user and the network resources, or specific applications that use and provide data to other devices attached to the network.

在此模型中，以太网完全适合链路层；IP和所有路由协议都适合互联网层；TCP 和 UDP 属于传输层。由于这种简洁性，这是一个用于理解当今部署的 IP 的更清晰的模型，尽管它没有提供划分各个级别信令的详细结构，而这在更加以研究为导向的环境中可能有用。

In this model, Ethernet fits wholly within the link layer; IP and all routing protocols fit in the Internet layer; and TCP and UDP fit within the transport layer. Because of this neatness, this is a cleaner model for understanding IP as its deployed today, although it doesn’t provide the detailed structure of splitting up the various levels of signaling that might be useful in a more research-oriented environment.

迭代模型

The Iterative Model

七层和四层模型依赖于这样的概念：当您将堆栈从物理层向上移动到应用程序时，每一层都会添加一些特定的功能或一组相关的功能。此功能通常与网络组件相关，例如单个链路、端到端路径或应用程序到应用程序的通信。通过将功能分组为层，可以最小化堆栈中每个协议的复杂性，并且可以定义和管理堆栈内每个协议或每个协议层之间的交互表面。

The seven- and four-layer models rely on the concept that as you move up the stack from the physical to the application, each layer adds some specific function, or a set of related functions. This functionality is generally tied to a component of the network, such as a single link, an end-to-end path, or application to application communication. By grouping functionality into layers, the complexity of each protocol in the stack can be minimized, and the interaction surface between each protocol or each layer of protocols within the stack can be defined and managed.

为了更全面地理解这些概念，查看网络中协议栈的最终模型很有用。虽然该模型不像七层和四层模型那样流行或广泛使用，但它更清楚地说明了复杂性和分层之间的权衡。

To understand these concepts a little more fully, it’s useful to look at one final model for protocol stacks in networks. While this model isn’t as popular or used as widely as the seven-and four-layer models, it more clearly illustrates the tradeoff between complexity and layering.

如果你检查一下上面协议栈中每一层的实际功能，你会发现它们之间有很多相似之处。例如，以太网数据链路层提供跨单个链路的传输和复用，而IP提供跨多跳路径的传输和复用。这导致了以下观察：任何数据承载协议实际上只有四个功能可以服务：传输、复用、纠错和流量控制。²这四种功能有两种自然分组：传输和复用、错误和流量控制。因此，大多数协议都会做以下两件事之一：

If you examine the actual function of each layer in the protocol stacks above, you’ll find many similarities between them. For instance, the Ethernet data link layer provides transport and multiplexing across a single link, and IP provides transport and multiplexing across a multihop path. This leads to the following observation: There are really only four functions that any data carrying protocol can serve: transport, multiplexing, error correction, and flow control.² There are two natural groupings within these four functions: transport and multiplexing, error and flow control. So most protocols fall into doing one of two things:

2 . John Day，网络架构模式：回归基础（新泽西州上萨德尔河；伦敦：Prentice Hall，2008 年）。

2. John Day, Patterns in Network Architecture: A Return to Fundamentals (Upper Saddle River, N.J.; London: Prentice Hall, 2008).

• 该协议提供传输，包括从一种数据格式到另一种数据格式的某种形式的转换，以及多路复用，该协议能够将来自不同主机和应用程序的数据分开。

• The protocol provides transport, including some form of translation from one data format to another, and multiplexing, the ability of the protocol to keep data from different hosts and applications separate.

• 该协议通过纠正小错误或重新传输丢失或损坏的数据的能力来提供错误控制，并提供流量控制，以防止由于网络传输数据的能力与应用程序生成数据的能力之间不匹配而导致的不当数据丢失。

• The protocol provides error control, either through the capability to correct small errors or to retransmit lost or corrupted data, and flow control, which prevents undue data loss because of a mismatch between the network’s capability to deliver data and the application’s capability to generate data.

从这个角度来看，以太网提供传输服务和流量控制，因此它是一个混合包，集中在网络内的单个链路、端口到端口（或隧道端点到隧道端点）上。IP 是一种提供传输服务的多跳协议（跨越多个物理链路的协议），而 TCP 是一种使用 IP 传输机制并提供纠错和流量控制的多跳协议。图 7.5说明了迭代模型。

From this perspective, Ethernet provides transport services and flow control, so it is a mixed bag concentrated on a single link, port to port (or tunnel endpoint to tunnel endpoint) within a network. IP is a multihop protocol (a protocol that spans more than one physical link) providing transport services, whereas TCP is a multihop protocol that uses IP’s transport mechanisms and provides error correction and flow control. Figure 7.5 illustrates the iterative model.

图 7.5 网络协议的迭代模型

Figure 7.5 Iterative Model of Network Protocols

该模型中的每一层将管理给定范围内的一组参数所需的信息分组到单个协议中。

Each layer in this model groups the information required to manage a single set of parameters within a given scope into a single protocol.

协议栈和设计

Protocol Stacks and Design

关于网络复杂性，特别是在设计领域，这一对协议设计和分层的小讨论告诉我们什么？首先，分层是一种经过时间考验、行之有效的方法，通过划分功能来将复杂性与复杂性分开跨越 API 边界。其次，三个模型的比较表明，将层与其所做的事情（而不是它们所在的位置）相匹配，可以提供更清晰且更易于处理的操作模型。虽然四层和七层模型有效，但它们既没有真正描述每一层的协议正在做什么，也没有真正建议将来将新协议插入堆栈的方法。另一方面，迭代模型提供了这两者。

What does this little excursion into protocol design and layering tell us about network complexity, particularly in the area of design? First, layering is a time tested, proven way to separate complexity from complexity by dividing functions across an API boundary. Second, the comparison of the three models shows that matching the layers to what they do, rather than where they are, provides a cleaner and easier to handle model of operations. While the four- and seven-layer models work, they neither really describe what the protocols at each layer are doing, nor do they really suggest a way to insert new protocols into the stack in the future. The iterative model, on the other hand, provides both of these.

协议复杂性与设计复杂性

Protocol Complexity versus Design Complexity

然而，协议栈不仅仅是独立的系统，它们还通过应用程序性能、可管理性和设计与更大的网络环境进行交互。本节更深入地重点讨论网络设计和协议复杂性之间的权衡，以使网络工程师更好地了解这些领域的权衡。具体来说，协议复杂性与设计复杂性如何权衡？

Protocols stacks aren’t just self-contained systems, however, they interact with the larger network environment through application performance, manageability, and design. This section focuses on the tradeoff between network design and protocol complexity a little more deeply, to give network engineers a better feel for the tradeoffs in these areas. Specifically, how does protocol complexity tradeoff against design complexity?

随着协议变得更加复杂，一些设计元素变得不那么复杂。另一方面，随着协议变得更加复杂，协议的故障排除和管理也变得更加复杂。每个动作都有一个相等且相反的反应，随着整个系统中每个组件的复杂性级别的上升和下降，创建一个动作和反应链。工程师需要意识到，一旦他们“把它扔到隔间墙上”，复杂性并不会“消失”。其他一些系统必须承担这种复杂性并对其进行管理。

As the protocol takes on more complexity, some elements of design become less complex. On the other hand, as the protocol becomes more complex, troubleshooting and managing the protocol becomes more complex as well. Each action has an equal and opposite reaction, creating a chain of action and reaction as the complexity levels of each component in the overall system rise and fall. Engineers need to be aware that the complexity doesn’t just “go away,” once they’ve “thrown it over the cubicle wall.” Some other system must pick that complexity up and manage it.

微循环和快速重新路由

Microloops and Fast Reroute

协议复杂性和网络复杂性之间的权衡的第一个例子是第 5 章“设计复杂性”中已经讨论过的概念。本节将深入探讨快速重路由的协议复杂性部分。这里将使用图7.6作为说明。

The first example of this tradeoff between protocol complexity and network complexity is a concept that’s already been discussed in Chapter 5, “Design Complexity.” This section will dig a little deeper in the protocol complexity pieces of fast reroute. Figure 7.6 will be used as an illustration here.

图 7.6 链路状态控制平面中的微循环

Figure 7.6 Microloops in a Link State Control Plane

假设该网络以以下状态开始：

Assume that this network starts with the state:

• 路由器 B 到 2001:db8:0:1::/64 的最短路径是经过 A。

• Router B’s shortest path to 2001:db8:0:1::/64 is through A.

• 路由器 C 到 2001:db8:0:1::/64 的最短路径是经过 B。

• Router C’s shortest path to 2001:db8:0:1::/64 is through B.

• 路由器 D 到 2001:db8:0:1::/64 的最短路径是经过 C。

• Router D’s shortest path to 2001:db8:0:1::/64 is through C.

• 路由器 E 到 2001:db8:0:1::/64 的最短路径是经过 A。

• Router E’s shortest path to 2001:db8:0:1::/64 is through A.

• 所有路由器都运行链路状态路由协议（OSPF 或IS-IS）。

• All routers are running a link state routing protocol (either OSPF or IS-IS).

当[B,C]链接失效时，会产生什么结果？要回答这个问题，请考虑：

When the [B,C] link fails, what will be the result? To answer this, consider:

• 路由器B 和C 将立即收到链路故障通知，并且本地进程将从本地路由表中删除受影响的目的地。

• Routers B and C will be notified of the link failure immediately, and local processes will remove the impacted destinations from the local routing table.

• 路由器B 和C 因为首先收到故障通知，所以将计算一棵新树并首先切换到新路由。

• Routers B and C, because they’ve been notified of the failure first, will calculate a new tree and switch to the new routes first.

• 路由器D 和A 将立即收到故障通知，计算新树并切换到新路由。

• Routers D and A will be notified of the failure second, calculate a new tree, and switch to the new routes.

• 路由器E 将最后收到故障通知，因此将最后计算并安装任何新的必要路由信息。

• Router E will be notified of the failure last, and hence will calculate and install any new necessary routing information last.

笔记

Note

在链路状态协议中，洪泛是与计算 SPT 分开的过程；为了方便起见，上面的描述同时考虑了它们两者。

In link state protocols, flooding is a separate process from calculating the SPT; for convenience, the description above considers them both at the same time.

结合网络的原始状态和[B,C]链路故障时的收敛顺序，可以很明显地看出微环发生在哪里。当Router C发现故障时，它会迅速重新计算SPT，发现经过Router D的路径是新的最短（无环）路径。当RouterC通过RouterD转发时，RouterD仍在发现故障并重新计算新的最短路径3。在时间差期间Router C 和 Router D 的重新计算，目的地为 2001:db8:0:1::/64 的流量将在两个路由器之间循环。

Combining the original state of the network with the order of convergence when the [B,C] link fails, it’s obvious where the microloop occurs. When Router C discovers the failure, it quickly recomputes the SPT and discovers that the path through Router D is the new shortest (loop free) path. While Router C is forwarding through Router D, Router D is still discovering the failure and recalculating a new shortest path three. During the differential in time between Router C’s recalculation and Router D’s, traffic destined to 2001:db8:0:1::/64 will be looped between the two routers.

为什么不设置一个计时器，让路由器 C 等到路由器 D 重新计算后安装新路由？这根本行不通。网络中的每个路由器必须以相同的方式对收到的信息做出反应；链路状态协议的全部要点是确保参与控制平面的每个设备都具有网络状态的单一视图，并且以相同的方式运行。这是 OSPF 和 IS-IS 等链路状态控制平面确定性的基础之一。在Router C 上设置等待收敛的定时器，就是在Router D 上设置同样的定时器。该解决方案所做的只是减慢整个网络的收敛速度，而不是真正解决微环路。

Why not set a timer so Router C waits until Router D has recalculated to install the new routes? This simply won’t work. Every router in the network must react the same way to received information; the entire point of a link state protocol is to ensure that every device participating in the control plane has a single view of the state of the network, and acts in the same way. This is one of the foundations of the determinism of link state control planes like OSPF and IS-IS. Setting up a timer during which Router C must wait to converge means setting the same timer on Router D; all this solution does is to slow the total network convergence down, rather than actually resolving the microloop.

笔记

Note

可以强制链路状态控制平面中的路由器命令安装前缀以防止微环；然而，它并不像使用计时器那么简单。相反，每个路由器必须计算其距网络变化的距离，并计算在安全地安装到本地路由表中每个目的地的新最短路径之前必须等待的时间。这个过程确实可以防止微循环，但代价是收敛速度。RFC6976 描述了有序转发信息库 (FIB) 解决方案，以及可以减轻某些收敛速度影响的消息完成系统。

It is possible to force the routers in a link state control plane to order the installation of the prefixes to prevent microloops; however, it’s not as simple as using timers. Instead, each router must calculate its distance from the network change, and calculate the amount of time it must wait before it can safely install new shortest paths to each destination in the local routing table. This process does prevent microloops, but at a cost in terms of convergence speed. RFC6976 describes the ordered Forwarding Information Base (FIB) solution, and a message of completion system that can alleviate some of the convergence speed impact.

为什么不强制链路状态网络中的所有路由器同时重新计算呢？当然，必须有一种方法来同步参与控制平面的所有设备的所有时钟，以便它们都同时开始计算，或者它们都同时安装其路由。但是，您能让数千个分布式节点上的所有设备上的所有计时器同步到什么程度呢？答案是：不能精确到微秒。即使你可以将所有时钟同步到这种精度，也几乎不可能确保每个路由器在计算后同时安装路由。有的路由器具有分布式转发平面，有的路由器则将转发与路由计算统一起来。在具有分布式转发平面的路由器中，每条路径在控制平面和转发硬件之间都有不同的路径，并且每条路径都将花费不同的时间来安装路由。即使在具有相同分布式系统的路由器中，每个路由器上的处理器负载也会随时变化，这再次影响了计算后安装路由所需的时间。

Why not just force all the routers in a link state network to recalculate at the same time? Surely there must be a way to synchronize all the clocks across all the devices participating in the control plane so they all begin calculating at the same time, or they all install their routes at the same time. But how closely can you get all the timers on all the devices across thousands of distributed nodes synchronized? The answer is: not down to the microseconds. Even if you could synchronize all the clocks to this level of precision, it would be almost impossible to ensure every router installs the route at the same time after the calculation. Some routers have distributed forwarding planes, while others unify forwarding with route calculation. Among routers that have distributed forwarding planes, each one is going to have different paths between the control plane and the forwarding hardware, and each of those paths are going to take a different amount of time to install the route. Even among routers with identical distributed systems, the processor load on each router will vary moment to moment, which again impacts the amount of time it takes to install a route once it’s been calculated.

以下各节将考虑该问题的几个答案。

Several answers to this problem are considered in the following sections.

无循环交替

无环路替代方案 (LFA) 可能在 JJ Garcia 的一篇论文中首次在学术上得到概述，该论文成为 Cisco EIGRP 的基础。³ LFA 的概念依赖于简单的几何结构，如图7.7所示。

Loop Free Alternates (LFAs) were probably first outlined academically in a paper by J. J. Garcia—the paper that became the foundation for Cisco’s EIGRP.³ The concept of an LFA relies on simple geometry, as shown in Figure 7.7.

3 . JJ Garcia-Lunes-Aceves，“使用扩散计算的无环路路由”，IEEE/ACM Transactions on Networking 1，第 1 期。1（1993 年 2 月）：130-141。

3. J. J. Garcia-Lunes-Aceves, “Loop-Free Routing Using Diffusing Computations,” IEEE/ACM Transactions on Networking 1, no. 1 (February 1993): 130–141.

图 7.7 无循环替代

Figure 7.7 Loop Free Alternates

在图7.7所示的网络中，RouterA到2001:db8:0:1::/64之间有3条路径：

In the network illustrated in Figure 7.7, there are three paths between Router A and 2001:db8:0:1::/64:

• [A,B,E]，度量为 4

• [A,B,E] with a metric of 4

• [A,C,E]，度量为 5

• [A,C,E] with a metric of 5

• [A,D,E]，度量为 6

• [A,D,E] with a metric of 6

要理解 LFA，还需要注意三个指标：

To understand LFAs, three more metrics need to be noted:

• [B,E] 度量为 2

• [B,E] with a metric of 2

• [C,E] 度量为 3

• [C,E] with a metric of 3

• [D,E] 度量为 4

• [D,E] with a metric of 4

LFA 的概念来自于这个简单的观察：“通过我”循环的任何路径的成本不能低于达到相同路径的“我的成本”目的地。换句话说，如果一条路径是“通过我”的环路，则邻居通告该路由的成本必须大于或等于到达同一目的地的本地最佳路径。

The concept of a LFA comes from this simple observation: the cost of any path that loops “through me” cannot be less than “my cost” to reach that same destination. To put this in other terms—if a path is a loop “through me,” the cost of the neighbor advertising the route must be more than or equal to the local best path to reach that same destination.

如图7.7所示，从RouterA的角度来看：

In terms of Figure 7.7, from the perspective of Router A:

• 如果从路由器A 到达2001:db8:0:1::/64 的路径成本大于4，则可能是环路。

• If the cost of a path to reach 2001:db8:0:1::/64 from Router A is greater than 4, then it might be a loop.

• 如果到达2001:db8:0:1::/64 的路径成本小于4，则它不可能是通过路由器A 本身返回的环路。

• If the cost of a path to reach 2001:db8:0:1::/64 is less than 4, then it cannot be a loop passing back through Router A itself.

有了这些信息，路由器 A 现在可以从每个连接的邻居的角度检查到达 2001:db8:0:1::/64 的成本，以确定它们到达该目的地的路径是否是通过路由器的环回A 本身与否。检查每个可用路径：

With this information in hand, Router A can now examine the cost to reach 2001:db8:0:1::/64 from the perspective of each of its connected neighbors to determine if their path to reach this destination is a loop back through Router A itself or not. Examining each of the paths available:

• (A,B,E)：这是最佳（成本最低）路径，因此不需要检查。

• (A,B,E): This is the best (lowest cost) path, so it does not need to be examined.

• (A,C,E)：Router C（Router A 的邻居）的 Cost 为 3，Router A 的最佳路径为 4。由于 Router C 的 Cost 小于 Router A 的 Cost，因此该路径不能通过 Router 环回A 本身。因此，该路径是无环路的（从路由器 A 的角度来看），因此是有效的无环路备用路径。

• (A,C,E): The cost at Router C (Router A’s neighbor) is 3, and the best path at Router A is 4. Because Router C’s cost is less than Router A’s cost, this path cannot loop back through Router A itself. Hence, this path is loop free (from Router A’s perspective), and is therefore a valid loop free alternate path.

• (A,D,E)：路由器 D 的成本为 4，路由器 A 的最佳路径为 4。由于这两个成本相等，因此流量有可能/可能转发到 2001:db8:0:1 ::/64 可以在某些拓扑更改事件期间转发回路由器 A（这是微循环的另一个实例）。因此，EIGRP 会将此路径声明为循环路径，而链路状态协议（OSPF 和 IS-IS）将确定这是一条通往目的地的可行的无循环备用路径。

• (A,D,E): The cost at Router D is 4, and the best path at Router A is 4. Because these two costs are equal, it is possible/probable that traffic forwarded to 2001:db8:0:1::/64 could be forwarded back to Router A during certain topology change events (this is another instance of a microloop). Because of this, EIGRP would declare this path as being a looped path, while link state protocols (OSPF and IS-IS) would determine this is a viable loop free alternate path to the destination.

笔记

Note

为了完整起见，路由器 A 处的最佳路径度量在 EIGRP 中称为“可行距离”，而路由器 B、C 和 D 处的度量则称为路由器 A 处的报告距离 — 因为这是成本（或距离）或度量）他们已向路由器 A 报告。

Just to be complete, the metric of the best path at Router A is called the Feasible Distance in EIGRP, and the metric at Routers B, C, and D is called the Reported Distance at Router A—because this is the cost (or distance or metric) they’ve reported to Router A.

从协议的角度来看，当使用 LFA 提供快速重新路由时，在复杂性方面有哪些权衡？

What are the tradeoffs, in terms of complexity, when using LFAs to provide fast reroute, from a protocol perspective?

•状态（携带）：控制平面协议本身没有携带额外的状态来提供无环路替代。对于链路状态协议，只需从邻居的角度运行 SPF，即可根据链路状态数据库计算到达邻居目的地的成本。对于 EIGRP，邻居的成本只是原始路由通告中包含的度量（不添加已连接链路的成本）。

• State (carried): There is no additional state carried in the control plane protocol itself to provide the loop free alternate. For link state protocols, the cost to reach a destination at a neighbor can be computed from the link state database by simply running SPF from the neighbor’s perspective. For EIGRP, the cost at the neighbor is simply the metric contained in the original route advertisement (without the cost of the connected links added in).

•状态（本地）：每个路由器或参与控制平面的每个设备内都有一些附加状态。对于链路状态协议，SPF 不仅必须在网络的本地视图上运行，而且还必须为每个邻居运行一次，以获得每个邻居到任何给定目的地的成本。备份路径在计算后还必须以某种方式存储，以便在主路径发生故障时转发平面可以快速切换到它。

• State (local): There is some additional state within each router, or each device participating in the control plane. For a link state protocol, SPF must be run not only for a local view of the network, but also once for each neighbor, to obtain each neighbor’s cost to any given destination. The backup path must also be somehow stored once calculated, so the forwarding plane can quickly switch over to it if the primary path fails.

•速度：理论上，快速重路由机制通过提高控制平面本地反应速度来降低控制平面对拓扑变化做出反应的全局速度。

• Speed: In theory, fast reroute mechanisms reduce the global speed at which the control plane must react to changes in the topology by increasing the speed at which they can react locally.

•表面：由于“在线”协议没有变化，因此当使用 LFA 进行快速重新路由时，控制平面和任何其他系统之间的表面几乎没有变化。

• Surface: As there are no changes to the protocol “on the wire,” there is little change in the surface between the control plane and any other system when using LFAs for fast reroute.

从协议的角度来看，复杂性权衡的负面结果很小，而正面结果却相当大。从设计角度来看（第 6 章“管理设计复杂性”中有更全面的介绍），LFA 不提供许多常用拓扑的覆盖范围。对于网络工程师决定是否部署 LFA，必须考虑两件事：

From a protocol perspective, the negative results of the tradeoff in complexity are minimal, while the positive results are fairly large. From a design perspective (covered more fully in Chapter 6, “Managing Design Complexity”), LFAs don’t provide coverage for many commonly used topologies. For the network engineer determining whether to deploy LFAs or not, two things must be considered:

• LFA 将覆盖网络的哪些部分，不覆盖哪些部分？

• Which parts of the network will LFAs cover, and which will they not cover?

• 部署LFA 来覆盖它们将覆盖的网络部分是否值得增加额外的复杂性？这里需要考虑的一个具体点是，如果网络的某些部分不会被覆盖，部署 LFA 可能不会以可测量的方式增加端到端路径的延迟/抖动特性（或者更确切地说，部署 LFA 不允许运营商要求固定的高速收敛时间）。

• Is it worth the additional complexity of deploying LFAs to cover the parts of the network they will cover? A specific point that needs to be considered here is that if there are parts of the network that will not be covered, deploying LFAs may not increase the delay/jitter characteristics of the end-to-end path in a measurable way (or rather, deploying LFAs won’t allow the operator to claim a fixed, high-speed convergence time).

关键点是从协议、配置、故障排除和管理角度考虑快速收敛、网络覆盖的业务需求和增加的复杂性之间的权衡。

The crucial point is to consider the tradeoff between business requirements for fast convergence, network coverage, and the added complexity at a protocol, configuration, troubleshooting, and management perspectives.

不通过

主拓扑 LFA 无法为环提供快速重路由，如图7.8所示。

The primary topology LFAs cannot provide fast reroute for is a ring, as illustrated in Figure 7.8.

图 7.8 用于快速重新路由的环形拓扑

Figure 7.8 A Ring Topology for Fast Reroute

在此网络中，路由器 A 有两条到 2001:db8:0:1::/64 的无环路路径，但在考虑 LFA 时，它会将经过路由器 B 的路径标记为潜在环路。RouterA如何使用路径[A,B,F,E]？问题是，在网络拓扑转换期间，路由器 A 发送到路由器 B 的任何流量（目的地为 2001:db8:0:1::/64 上的主机）可能会被路由器 B 环回（请参阅第 5 章， “设计复杂性，”了解更多详细信息）。如何解决这个问题呢？如果路由器 A 可以弄清楚如何将流量通过隧道传输到始终指向路由器 E 的路由器以到达 2001:db8:0:1::/64，那么它可以在拓扑更改期间通过此隧道转发流量，并且仍然可以确定流量不会循环（将到达正确的目的地）。

In this network, Router A has two loop free paths to 2001:db8:0:1::/64, but it will mark the path through Router B as a potential loop when considering LFAs. How can Router A use the path [A,B,F,E]? The problem is that during transitions in the network topology, any traffic Router A sends to Router B, destined to a host on 2001:db8:0:1::/64, might be looped back by Router B (see Chapter 5, “Design Complexity,” for more details). How can this problem be solved? If Router A can figure out how to tunnel the traffic to a router that always points to Router E to reach 2001:db8:0:1::/64, then it can forward traffic over this tunnel during topology changes and still be certain the traffic will not loop (will reach the correct destination).

路由器 A 如何找出满足此条件的潜在隧道端点？NotVia 是一种解决方案。假设网络管理员确定需要保护链路 [D,E] 免遭故障。以相当简化的形式：

How can Router A find out about potential tunnel endpoints that meet this criteria? NotVia is one solution. Assume that the network administrator determines the link [D,E] needs to be protected from failure. In a rather simplified form:

• 路由器E 配置有特殊的IP 地址。该地址称为E，而不是 D，以供本说明全文参考。

• Router E is configured with a special IP address. This address is called E notvia D for reference throughout this explanation.

• E notvia D由路由器 E 通告给除路由器 D 之外的所有其他邻居。在这种情况下，E notvia D仅通告给路由器 F。

• E notvia D is advertised by Router E to every other neighbor than Router D. In this case, E notvia D is advertised only to Router F.

• 路由器A 仅从路由器B 接收此通告，无论[A,D,E] 路径的状态如何，都会为其提供一条到路由器E 的路由。

• Router A receives this advertisement only from Router B, giving it a route to Router E no matter what the state of the [A,D,E] path is.

• 当路由器A 计算到2001:db8:0:1::/64 的路径时，它首先找到沿[A,D,E] 的最佳（最低成本）路径，并将其安装到本地表中。

• When Router A calculates the path to 2001:db8:0:1::/64, it first finds the best (lowest cost) path along [A,D,E], and installs this in the local table.

• 然后，路由器A 搜索到达该目的地的备用路径。发现Router B在路径中，实际上有一条路由E notvia D，它安装notvia路由作为到目的地的备份隧道路径。

• Router A then searches for an alternate path to this destination. Finding Router B is in the path, and there is, in fact, a route E notvia D, it installs the notvia route as a backup tunnel path to the destination.

• 如果路径[A,D,E] 出现故障，路由器A 会将目的地为2001:db8:0:1::/64 的流量切换到隧道路径。流量封装在 [A,B,F,E] 后面的标头中。路由器 E 删除该外部报头并根据数据包内部的信息进行转发。由于隧道数据包中的内部标头的目的地是 2001:db8:0:1::/64 上的主机，因此数据包将从直接连接的接口转发出去。

• If the path [A,D,E] fails, Router A switches traffic destined to 2001:db8:0:1::/64 to the tunneled path. Traffic is encapsulated in a header that follows [A,B,F,E]. Router E removes this outer header and forwards based on the information inside the packet. Because the inner header in the tunneled packet is destined to a host on 2001:db8:0:1::/64, the packet is forwarded out the directly connected interface.

笔记

Note

此序列假定从路由器 A 到路由器 E 的最佳路径是沿着 [A,B,F,E]。因为情况可能并非如此，NotVia 会改为计算到路由器 F。如何做到这一点超出了这个概念的简单解释的范围。

This sequence assumes that the best path to Router E from Router A is along [A,B,F,E]. Because this may not be the case, NotVia would calculate to Router F instead. How this is done is beyond the scope of this simple explanation of the concept.

从复杂性的角度来看，NotVia 的表现如何？

How does NotVia fare from a complexity perspective?

•状态（携带）：NotVia 需要为网络中的每个受保护链路或节点提供额外的IP 地址。这可能（取决于有多少链路或设备受到保护）是注入控制平面的大量附加 IP 地址，并且这些地址必须在协议中以某种方式标记为 notvia 地址，因此它们不用于正常转发。因此，NotVia 增加了状态。事实上，这种附加状态是 NotVia 被拒绝作为互联网标准的主要原因。

• State (carried): NotVia requires an additional IP address for each protected link or node in the network. This may (depending on how many links or devices are protected) be a large number of additional IP addresses injected into the control plane—and these addresses must be somehow marked in the protocol as notvia addresses, so they’re not used for normal forwarding. State is, therefore, increased with NotVia. In fact, this additional state is the primary reason NotVia was rejected as an Internet Standard.

•状态（本地）：参与控制平面的每个设备都需要保留额外的notvia地址，并计算包含这些链路的本地 SPT，以发现备用路由。计算时间可能很短，但有一些额外的状态。

• State (local): Each device participating in the control plane will need to keep the additional notvia addresses, and calculate their local SPT with these links included to discover alternate routes. The calculation time is likely minimal, but there is some additional state.

•速度：理论上，快速重路由机制通过提高控制平面本地反应速度来降低控制平面对拓扑变化做出反应的全局速度。

• Speed: In theory, fast reroute mechanisms reduce the global speed at which the control plane must react to changes in the topology by increasing the speed at which they can react locally.

•表面：虽然协议发生了变化，但控制平面与其交互的其他网络系统之间的交互表面几乎没有变化。NotVia确实加深了路由器之间的交互面，然而，有两种方式。首先，控制平面中必须携带、计算和依赖额外的状态。其次，在控制平面中托管NotVia地址的任何设备上都添加了“开放隧道” 。

• Surface: Although there are protocol changes, there is little change in the interaction surfaces between the control plane and the other network systems it interacts with. NotVia does deepen the interaction surfaces between routers, however, in two ways. First, there is additional state that must be carried, computed, and relied on in the control plane. Second, there is the addition of an “open tunnel” on any device in the control plane hosting a NotVia address.

NotVia 确实向控制平面添加了一些状态，因此确实增加了控制平面的复杂性。

NotVia does add some state to the control plane, and hence does increase control plane complexity.

远程无环路替代

NotVia 的替代方案是路由器 A（如图 7.8所示）使用某种其他机制来计算将到达 2001:db8:0:1::/64 的远程下一跳，即使其通过路由器 D 的主路径已发生故障，并且找到一些替代方法将数据包通过隧道传送到该中间路由器。远程 LFA 正好提供了这样的解决方案。要计算远程 LFA：

An alternative to NotVia is for Router A, in Figure 7.8, to use some other mechanism to calculate a remote next hop that will reach 2001:db8:0:1::/64 even if its primary path through Router D has failed, and find some alternate way to tunnel packets to this intermediate router. Remote LFAs provide just such a solution. To calculate a remote LFA:

• 路由器A 从其邻居的邻居的角度计算SPT 。在这种情况下，路由器 A 将从路由器 F 的角度计算到 2001:db8:0:1::/64 的最佳路径。

• Router A calculates a SPT from its neighbor’s neighbor’s perspective. In this case, Router A would calculate the best path to 2001:db8:0:1::/64 from the perspective of Router F.

• 发现Router F 实际上有到目的地的无环路径，Router A 就可以建立一条到Router F 的隧道，并将该隧道作为通过Router D 的主路由的备份。

• Finding Router F does, in fact, have a loop free path to the destination, Router A can then build a tunnel to Router F, and install this tunnel as a backup to the primary route through Router D.

在这个解决方案中，主要问题是应该使用什么形式的隧道来构建这些备用路径。这个问题最常见的答案是 MPLS，因为它是轻量级的，并且具有动态构建隧道所需的所有信令。鉴于某种类型的 MPLS 信令已因其他原因在网络中运行，从网络管理的角度来看，添加远程 LFA 是一项相当简单的工作。当然，动态创建的隧道可以为故障排除过程增添一些乐趣；可能需要花费一些时间和仔细的工作来弄清楚为什么特定流通过动态隧道的网络中的特定链路传输，因为各个设备上的配置与创建的实际隧道之间没有联系。

The main question, in this solution, is what form of tunneling should be used to build these alternate paths. The most common answer to this question is MPLS, as it is lightweight, and has all the signaling necessary to dynamically build tunnels. Given MPLS signaling of some type is already running in the network for other reasons, adding Remote LFAs is a fairly trivial exercise from a network management perspective. Dynamically created tunnels can add some excitement to the troubleshooting process, of course; it can take some time and careful work to sort out why specific flows are traveling over specific links in the network with dynamic tunnels, because there’s no connection between the configurations on individual devices and the actual tunnels created.

同样，值得研究的是远程 LFA 与此处使用的复杂性模型的比较。

Again, it’s worth examining how Remote LFAs compare to the complexity model used here.

•状态（携带）：远程LFA 不会专门向路由协议添加任何新信息，但它们确实需要某种方法来通过网络构建备份隧道。如果某种形式的动态隧道系统已经在网络上运行（例如，支持 MPLS 虚拟专用网络或流量工程/MPLS-TE），那么这个额外的状态很小。如果必须部署某种形式的动态隧道信令来支持远程 LFA，那么额外的控制平面状态以及复杂性的增加可能会令人望而生畏。

• State (carried): Remote LFAs don’t add any new information to the routing protocol specifically, but they do require some way to build backup tunnels through the network. If some form of dynamical tunneling system is already running on the network (for instance, to support MPLS virtual private networks or traffic engineering/MPLS-TE), then this additional state is very small. If some form of dynamic tunneling signaling must be deployed to support Remote LFAs, then the additional control plane state—and increase in complexity—could be daunting.

•状态（本地）：参与控制平面的每个设备都需要执行一组附加的 SPF 计算，并存储有关备份隧道的信息。当然，还必须维护备份隧道的状态，因此这很可能是大量状态（因此，复杂性很高）。

• State (local): Each device participating in the control plane will need to perform an additional set of SPF calculations, and to store information about a backup tunnel. Of course, the state of the backup tunnel must be maintained, as well, so this could well be a large amount of state (and hence, a large amount of complexity).

•速度：理论上，快速重路由机制通过提高控制平面本地反应速度来降低控制平面对拓扑变化做出反应的全局速度。

• Speed: In theory, fast reroute mechanisms reduce the global speed at which the control plane must react to changes in the topology by increasing the speed at which they can react locally.

•表面：远程LFA 向网络添加至少一个新系统，这是一种动态构建和管理备用路径隧道覆盖的机制。这增加了几个新的交互表面，例如控制平面和隧道信令系统之间的接口，以及物理拓扑和隧道覆盖。如果由于其他原因部署了动态隧道机制，那么这些额外的交互表面已经就位，并且网络中的复杂性不会增加。然而，如果部署动态隧道来支持远程 LFA，则它们的部署将增加网络中交互表面的数量。

• Surface: Remote LFAs add at least one new system to the network, some mechanism to dynamically build and manage an overlay of alternate path tunnels. This adds several new interaction surfaces, such as the interface between the control plane and the tunnel signaling system, and the physical topology and the tunneled overlay. If a dynamic tunneling mechanism has been deployed for some other reason, then these additional interaction surfaces are already in place, and complexity isn’t increased in the network. If dynamic tunneling is being deployed to support Remote LFAs, however, their deployment will increase the number of interaction surfaces in the network.

笔记

Note

此权衡列表假设标签分发协议 (LDP) 实际上在构建备份隧道所需的所有节点上运行，因此添加远程 LFA 的成本主要在于网络中携带的状态量，以及额外的隧道、端点等。事实上，如果部署远程 LFA 需要将 LDP 部署到网络中更广泛的节点（在大多数网络设计中这是一种明显的可能性），则需要提高网络性能，或者业务驱动因素需要更多坚持，以证明在其他节点上部署 LDP 的合理性。在考虑复杂性权衡时，应始终包括将协议和解决方案部署到网络中其他节点的成本。

This list of tradeoffs assumes that Label Distribution Protocol (LDP) is, in fact, running on all the nodes required to build backup tunnels, so the cost to add Remote LFAs is primarily in the amount of state carried in the network, as well as additional tunnels, endpoints, etc. In fact, if deploying Remote LFAs requires deploying LDP to a wider set of nodes in the network—a distinct possibility in most network designs—the increased network performance would need to be greater, or the business drivers more insistent, to justify deploying LDP on the additional nodes. The cost of deploying protocols and solutions onto additional nodes in the network should always be included when considering the complexity tradeoffs.

EIGRP 和设计难题

EIGRP and the Design Conundrum

在大规模网络时代的早期，EIGRP 作为一种路由协议而享有盛誉，您可以“将其应用于任何拓扑，无需大量设计工作，并且会正常工作”。事实上，这种信念有很多道理——事实上，这种说法有点太真实了。很快，很大在运行 EIGRP 时，计划外的网络开始出现严重故障，从而得出相反的结论：EIGRP 是一个非常糟糕的控制平面。在某种程度上，EIGRP 是其早期成功支持大型复杂网络而很少考虑实际设计的受害者。

Early in the era of large-scale networking, EIGRP gained a reputation as being a routing protocol you could “throw onto any topology, without a lot of design effort, and will just work.” There is, in fact, a good deal of truth to this belief—in fact, there’s a bit too much truth to this statement. Soon enough, very large unplanned networks began failing in a big way while running EIGRP—leading to the opposite conclusion, that EIGRP is a really horrible control plane. In a way, EIGRP is a victim of its own early success in supporting large complex networks with little thought to actual design.

为什么 EIGRP 在几乎没有设计工作的情况下就能在大型网络中表现得如此出色？使用图 7.9中的拓扑对 EIGRP 操作进行简短回顾可能有助于解释。

Why did EIGRP hold up so well in large-scale networks with little design effort? A short review of EIGRP operation might be helpful in explaining, using the topology in Figure 7.9.

图 7.9 EIGRP 操作概述

Figure 7.9 EIGRP Operation Overview

如果路由器 A 失去与 2001:db8:0:1::/64 的链接，会发生什么情况？

If Router A loses its link to 2001:db8:0:1::/64, what happens?

1. Router A 首先检查其本地信息以确定该目的地是否存在 LFA（ EIGRP 中的FS ）。

1. Router A will first examine its local information to determine if there is an LFA (a FS in EIGRP) for this destination.

2.如果没有找到，它将将该路由标记为活动（这意味着 EIGRP 正在主动寻找到该目的地的备用路由），并向其每个邻居发送查询，在本例中为路由器 D。

2. Finding none, it will mark the route active (which means EIGRP is actively looking for an alternate route to this destination), and send a query to each of its neighbors, in this case Router D.

3.路由器 D 在收到此查询后，将检查其本地表并发现路由器 A 是 2001:db8:0:1::/64 可用的唯一路径。路由器 D 会将该路由标记为活动状态，并向路由器 E发送查询。

3. Router D, on receiving this query, will examine its local tables and discover Router A is the only path available to 2001:db8:0:1::/64. Router D will mark this route active, and send a query to Router E.

4.路由器E将效仿路由器D，向路由器F发送查询。

4. Router E will follow Router D’s example, sending a query to Router F.

5. Router F 将按照 Router E 的示例，向 Router G发送查询。

5. Router F will follow Router E’s example, sending a query to Router G.

6.路由器 G 会发现它没有邻居可供查询，因此它将标记 2001:db8:0:1::/64 unreachable，并向路由器 F 发送回复，表明它没有到达目的地的备用路径。

6. Router G will find it has no neighbors to query, so it will mark 2001:db8:0:1::/64 unreachable, and send a reply to Router F stating it has no alternate path to the destination.

7.路由器F收到此回复后，将标记该路由不可达，并向路由器E发送回复，表明它没有到达此目的地的备用路径。

7. Router F, on receiving this reply, will mark the route unreachable, and send a reply to Router E stating it has no alternate path to this destination.

8.此回复链将继续，直到路由器 A 收到来自路由器 D 的回复。此时，该路由已从路由器 A 的路由表中删除，并且路由器 A 发送以下更新，其中包含 2001:db8:0 的不可达度量值： 1::/64。这将从其他路由器的表中删除该目的地。

8. This chain of replies will continue until Router A receives a reply from Router D. At this point, the route is removed from Router A’s routing table, and Router A sends a following update with an unreachable metric for 2001:db8:0:1::/64. This removes the destination from the other router’s tables.

笔记

Note

在如图 7.9所示的简单网络中，整个查询过程似乎是多余的，甚至“不是很有用”。实际上，EIGRP 中的整个查询机制旨在查找先前标记为环路或由于水平分割而未报告的备用路径。因此，EIGRP 中的查询过程的行为非常类似于链路状态协议中用于发现远程 LFA 的机制，只是它作为正常收敛的一部分内置到协议中，而不是作为快速重新路由机制添加。

In simple networks like the one shown in Figure 7.9, this entire query process seems redundant—and even “not very useful.” In reality, the entire query mechanism in EIGRP is designed to find alternate paths previously marked as loops, or not reported because of split horizon. As such, the query process in EIGRP acts much like the mechanisms used in a link state protocol to discover a remote LFA—only it’s built into the protocol as a part of normal convergence, rather than added on as a fast reroute mechanism.

这个过程有几个有趣的点值得注意：

There are several interesting points about this process worth noting:

• 通过此过程携带的状态量是最少的。由于EIGRP是距离矢量协议，不携带拓扑信息，仅携带可达性信息。

• The amount of state being carried through this process is minimal. As EIGRP is a distance vector protocol, no topology information is carried, only reachability information.

• 通过这种扩散更新过程，EIGRP 将查找任何可用备用路径的负载分散到故障域内的每个路由器。这种负载分散机制使 EIGRP 对于大规模环境非常稳健（可能有 50 万条或更多路由，有数百个邻居，尤其是在中心辐射型拓扑中）。

• Through this diffusing update process, EIGRP spreads the load of finding any available alternate paths to every router within the failure domain. This load spreading mechanism makes EIGRP extremely robust to large-scale environments (half a million routes or more is possible, with hundreds of neighbors, especially in hub-and-spoke topologies).

• 发现任何备用路由的过程类似于在树本身上实时运行SPF。由于这是以串行方式执行的，因此 EIGRP 中不会形成微循环。数据包在收敛期间将被丢弃，但不会循环。

• The process of discovering any alternate routes is similar to running SPF on the tree itself, in real time. Because this is performed in a serialized way, no microloops are ever formed in EIGRP. Packets will be dropped during convergence, but never looped.

那么为什么 EIGRP 在相对设计自由的网络中能够如此良好地扩展呢？因为与大多数其他协议相比，控制平面中携带的状态最少，并且扩散更新算法 (DUAL) 在通过网络分散收敛负载方面做得很好。

So why does EIGRP scale so well in relatively design free networks? Because the state carried in the control plane is minimal compared to most other protocols, and because the Diffusing Update Algorithm (DUAL) does such a good job at spreading the convergence load through the network.

那么，为什么 EIGRP 网络会出现故障呢？EIGRP 可能会遇到几种特定的网络情况，这些情况会导致某种类型的收敛失败，包括：

Why, then, do EIGRP networks fail? EIGRP can encounter several specific network situations that will cause a convergence failure of some type, including:

• 极长的“查询尾部”。只要查询本身不包含目的地的本地知识，EIGRP 查询就会停止，这通常意味着“网络边缘”，或者聚合可达性信息的某个地方。如果网络中没有配置聚合，则每个查询都将由网络中的每个路由器处理。如果单个链路发生故障，就会导致数千个目的地无法访问当 EIGRP 脱离网络时，使 EIGRP 稳健的分布式收敛过程实际上与协议背道而驰，导致比使网络收敛所需的工作多得多。

• Extremely long “query tails.” An EIGRP query stops wherever there is no local knowledge of the destination contained in the query itself—this generally means “the edge of the network,” or someplace where reachability information is aggregated. If there is no aggregation configured in the network, each and every query will be processed by every router in the network. If a single link fails that causes thousands of destinations to be dropped off the network, the distributed convergence process that makes EIGRP robust actually works against the protocol, causing a lot more work than is necessary to bring the network to convergence.

• 大量并行路径。由于每个 EIGRP 路由器都会向其每个邻居发送查询，因此大量并行路径会导致通过网络传输大量查询。这使 EIGRP 状态机工作非常困难，再次导致 DUAL 违背其更好的属性。

• High numbers of parallel paths. Because each EIGRP router sends a query to each of its neighbors, a high number of parallel paths cause a large number of queries to be transmitted through the network. This works the EIGRP state machine very hard, again causing DUAL to work against its better attributes.

• 内存量或处理性能较低的网络设备与具有大处理器和大量内存的设备配对。网络中一组不匹配的设备意味着一个路由器可以快速向根本无法处理处理负载的邻居发送数千个查询。在这种情况下，DUAL 过程再次开始发挥作用。

• Network devices with low amounts of memory or processing performance paired with devices that have big processors and lots of memory. A mismatched set of devices in the network means one router can quickly send thousands of queries to a neighbor that simply cannot handle the processing load. Again, the DUAL process begins to work against itself in these types of situations.

这些情况最明显的结果就是卡在活动路线中。当 EIGRP 网络中的每个路由器发送查询时，它都会设置一个计时器；路由器期望在这段时间内收到该查询的答复。如果在此时间段内没有收到回复，则该路由被声明为处于 active 状态，并且与未收到回复的邻居的邻接关系将被重置。

The most obvious result of any of these situations is a stuck in active route. As each router in the EIGRP network transmits a query, it sets a timer; the router expects to receive an answer for this query within this period of time. If no reply is received within this time period, the route is declared stuck in active, and the adjacency with the neighbor for which no reply was forthcoming will be reset.

正如您可以想象的那样，当协议已经承受大规模收敛事件的压力时，您最不想做的事情就是重置邻居邻接关系，从而导致另一轮查询，从而给网络带来更大的压力。为什么协议设计成这样工作？因为，实际上，一旦定时器启动，控制平面就已经脱离了 DUAL 定义的有限状态机。除了重置状态机之外，没有真正的方法可以解决该问题。

As you can imagine, the last thing you want to do when the protocol is already under the stress of a large-scale convergence event is to reset the neighbor adjacencies, causing another round of queries, and hence more stress in the network. Why was the protocol designed to work this way? Because, in effect, once the timer has fired the control plane has stepped outside the finite state machine defined by DUAL. There is no real way to resolve the problem other than to reset the state machine.

多年来，当许多工程师发现网络中的活动路由卡住时，他们只是简单地增加卡住活动计时器。然而，这恰恰是解决问题的错误方法。如果您将卡在活动计时器中的时间视为您愿意允许网络保持不收敛的时间量（因此将数据包丢弃到实际上应该可达的目的地），您可以立即看到以下不良副作用增加这个计时器。相反，网络和协议都需要共同努力来解决这种情况。

For years, many engineers simply increased the stuck in active timer when they would see stuck in active routes in their network. This is, however, precisely the wrong solution to the problem. If you view the stuck in active timer as the amount of time you’re willing to allow the network to remain unconverged (and hence dropping packets to a destination that should, actually, be reachable), you can immediately see the bad side effects of increasing this timer. Instead, the network and protocol both need to work to resolve this situation.

在大规模环境中使用该协议的实际经验揭示了这些问题。作为回应，采取了两条行动：

Real field experience with the protocol in large-scale environments revealed these problems. In response, two lines of action were taken:

• EIGRP 作为一种协议，几乎可以在您可以配置它的任何东西上运行，这一看法已转变为更加平衡的观点。事实上，EIGRP 可以容忍各种拓扑和网络状况。然而，仍然需要考虑一些具体的设计参数，例如故障域的大小、部署的设备的质量以及其他因素。在复杂性领域，这实际上将协议处理的一些复杂性“抛回了隔间墙”，由网络设计者来处理。这里有一个关于平衡复杂性位置的重要教训。

• The perception of EIGRP as a protocol that would run over just about anything you could configure it on was changed to a more balanced view. EIGRP will, in fact, tolerate a wide variety of topologies and network conditions. However, there still needs to be thought put into some specific design parameters, such as the size of a failure domain, the quality of the equipment deployed, and other factors. In the realm of complexity, this is effectively throwing some of the complexity being handled by the protocol “back over the cubicle wall,” to be handled by the network designer. There is an important lesson in balancing the location of complexity here.

• EIGRP停留在活动进程中的情况已被修改，以允许协议更优雅地在 DUAL 状态机内停留更长时间，同时减少对实际网络操作的影响。具体来说，陷入活跃状态的到期计时器导致通过查询链传输新的查询，而不是重置邻居邻接关系。这使得具有较小处理器和内存池的路由器有足够的时间来处理大量查询或解决本地邻接问题，而不会影响整个网络。从复杂性角度来看，这组添加的数据包和计时器增加了协议的复杂性，以解决实际部署中出现的一些问题。在这里，部署的复杂性与协议的复杂性进行了权衡，因为协议的复杂性似乎并不像重新设计全球每个 EIGRP 网络那样难以管理。

• EIGRP’s stuck in active process was modified to allow the protocol to more gracefully remain inside the DUAL state machine for longer while decreasing the impact on the actual network operation. Specifically, the expiration of the stuck in active timer caused a new query to be transmitted through the query chain, rather than resetting the neighbor adjacency. This allowed routers with smaller processors and memory pools the time they needed to process large groups of queries, or a local adjacency problem to be sorted out without impacting the network as a whole. In complexity terms, this added set of packets and timers increased the complexity of the protocol to resolve some of the problems being seen in actual deployments. Here, complexity in deployment was traded off against protocol complexity, as the protocol complexity didn’t appear to be as difficult to manage as redesigning every EIGRP network in the world wholesale.

通过多年的经验，EIGRP 的开发和修改表明了在现实世界中必须如何处理复杂性权衡。有时最好将复杂性转移到其他地方，有时最好将复杂性保留在协议中。这完全取决于工程师面临的具体情况。

The development, and modification, of EIGRP through years of experience shows how complexity tradeoffs must be handled in the real world. Sometimes it’s best to move the complexity someplace else, sometimes it’s best to keep the complexity in the protocol. It all depends on the specific situation the engineer faces.

最后的想法

A Final Thought

协议、设计和系统复杂性的交叉本身就是一个复杂的话题。本章实际上只是触及了这个广泛而有趣的研究领域的表面。然而，这里的例子和概述的想法应该为你提供坚实的工作知识核心，一套工具，至少可以帮助你认识到任何给定情况下的权衡，并三思而行“把复杂性扔到隔间墙上”。 ”

The intersection of protocol, design, and system complexity is itself a complex topic. This chapter really only scratches the surface of this broad and interesting area of investigation. However, the examples and outlined thoughts here should provide you with a solid core of working knowledge, a set of tools that will help you at least recognize the tradeoffs in any given situation, and to think twice about “tossing complexity over the cubicle wall.”

第 8 章复杂系统如何失败

Chapter 8. How Complex Systems Fail

每个幸福的家庭都以大致相同的方式幸福，但每个不幸福的家庭都以完全独特的方式痛苦。

Every happy family is happy in much the same way, but every unhappy one is miserable in completely unique ways.

只需稍加改动即可将其应用到网络世界中：“每个成功的网络都以与世界上其他网络大致相同的方式实现这一点，但每个失败的网络都以完全独特的方式实现这一点。” 扭转这一局面，这意味着学习网络设计的最佳方法之一就是解决失败问题。工程师接触到的每一次失败的网络设计也会给设计带来一些教训。

To apply this to the network world with just a slight change—“every network that succeeds does so in much the same way as every other network in the world, but every network that fails does so in a completely unique way.” Turning this around, this means one of the best ways to learn network design is to work on failures. Every failed network design an engineer touches teaches some lesson in design, as well.

但要真正学习网络工程，需要一点故障理论——对所谓的“故障理论”的理解提供了一个背景或一个框架，工程师不仅可以将故障放入其中，还可以提出新的想法。通过了解要寻找什么样的故障模式，然后实际寻找，可以有效地评估协议、网络设备和网络作为一个系统的新（和旧）想法的潜在故障。网络工程中经常发生的事情是找到需要解决的问题，然后解决它，而不考虑复杂性权衡，也不考虑协议或网络中可能导致的故障模式。

But to really learn network engineering, a little theory of failure is called for—an understanding of what might be called “failure theory” provides a context, or a framework, into which an engineer cannot only place failures, but also new ideas. New (and old) ideas in protocols, network devices, and networks as a system can be effectively evaluated for potential failures by knowing what sorts of failure modes to look for—and then actually looking. What tends to happen in network engineering is finding a problem that needs to be solved, and then solving it—without thought for the complexity tradeoffs, and without considering the failure modes that might result up in the protocol or network.

潜在故障模式的整个概念与意外后果的概念以及复杂性曲线左下角的不可到达空间（或 CAP 定理的不可填充三角形，早在第 1 章“定义复杂性”中讨论过）一起工作。。

This entire concept of potential failure modes works alongside the concept of unintended consequences, and the unreachable space at the bottom left corner of the complexity curve (or the unfillable triangle of the CAP theorem, discussed way back in Chapter 1, “Defining Complexity”).

本章将从理论角度讨论网络失败的两个主要原因：

This chapter will discuss two major reasons why networks fail from a theoretical perspective:

• 正反馈循环

• Positive feedback loops

• 命运共同体

• Shared fate

这些类别可能看起来很广泛，但本章将通过一些示例来充实它们，这些示例应该可以帮助您了解在每种情况下要寻找的内容。还将检查这些故障模式，以了解它们如何与其他常见的网络故障原因相互作用，例如微调控制平面的速度或重新分配。

These categories might seem broad, but this chapter will flesh them out with a number of examples that should help you understand what to look for in each case. These failure modes will also be examined to see how they interact with other commonly asserted causes of network failure, such as the speed of a finely tuned control plane, or redistribution.

反馈回路

Feedback Loops

反馈环路可用于许多不同的用途，例如，在锁相环路中，电路的反馈经过仔细控制以产生恒定频率波形或载波。事实上，锁相环构成了几乎所有现代无线电系统的基础。反馈回路的另一个例子是控制进入内燃机和喷气推进发动机的空气和燃料流的各种类型的机构。如果没有这些反馈回路，这些类型的发动机只能以非常低的水平运行，并且效率较低。

Feedback loops are useful for a lot of different things—for instance, in a phase locked loop, the feedback of the circuit is carefully controlled to produce a constant frequency waveform or carrier. Phased locked loops form the foundation, in fact, of almost all modern radio systems. Another example of a feedback loop is the various types of mechanisms controlling the air and fuel flow into internal combustion and jet propulsion engines. Without these feedback loops, these types of engines can only run at a very minimal level, and at a somewhat low efficiency.

反馈有两种：负面反馈和正面反馈。图 8.1使用非常简单的振荡信号方式解释了负反馈环路。

There are two kinds of feedback: negative and positive. Figure 8.1 illustrates a negative feedback loop using a very simple oscillating signal style of explanation.

图 8.1 负反馈环路

Figure 8.1 A Negative Feedback Loop

在该图中，有两个设备；您无需了解此处所示的组件实际上如何工作即可理解反馈循环的概念：

In this figure, there are two devices; you don’t need to understand how the components illustrated here actually work to understand the concept of a feedback loop:

•添加：该设备仅获取两个输入信号并将它们组合起来以创建单个输出信号。

• Add: This device simply takes the two input signals and combines them to create a single-output signal.

• Tap：该设备仅复制呈现给一个输入的信号，进行任何编程修改，然后在其第二个输出接口上输出结果。

• Tap: This device simply replicates the signal presented to one input, does any programmed modification, and outputs the result on its second output interface.

在这种情况下，TAP 设备被配置为反转信号并稍微降低其强度（更准确地说是幅度）。该插图中发生的情况是这样的：

In this case, the TAP device is configured to invert the signal and reduce its strength (amplitude in more precise terms) a bit. What happens in this illustration is this:

1.标记为 A 的波形部分穿过 ADD 器件。

1. The section of the wave marked A passed through the ADD device.

2.起初，第二个输入上没有信号，因此波没有变化地通过 ADD 器件。

2. At first, there is no signal on the second input, so the wave passes through the ADD device without change.

3.波继续前进并进入 TAP 设备。

3. The wave continues on and passes into the TAP device.

4.在 TAP 设备处，制作波的一个小副本并反转（因此正峰值与负峰值反转），并且信号的这个小副本沿着路径传回到 ADD 的第二个输入设备。

4. At the TAP device, a small replica of the wave is made and inverted (so the positive peaks are reversed with the negative peaks), and this little replica of the signal is passed back along the path to the second input of the ADD device.

5.从TAP设备反馈的该信号由ADD设备添加到原始信号中。

5. This signal, being fed back from the TAP device, is added to the original signal by the ADD device.

6.因为该信号是反相的，所以两个波的相加具有降低ADD设备的输出强度的效果。这个过程很像取一个大数，将其乘以某个非常小的数（例如 0.01），反转该数（将其从正号变为负号），然后将其加回到原始数。

6. Because this signal is inverted, the addition of the two waves has the effect of reducing the strength of the output of the ADD device. This process is much like taking a large number, multiplying it by some very small number (say 0.01), inverting the number (changing it from a positive sign to a negative), and adding it back to the original number.

7.这会导致 ADD 设备的输出电平降低。

7. This has the effect of causing the output level of the ADD device to decrease.

8.然后，该降低的信号被反馈到 TAP 设备，TAP 设备采用相同比例的接收信号，将其反转，并将其反馈回 ADD 设备。

8. This decreased signal is then fed back into the TAP device, which takes the same proportion of the received signal, inverting it, and feeding it back to the ADD device.

正如您从插图右侧的输出信号中看到的，结果是信号强度不断下降。人们很容易认为这种情况会稳定下来，但它永远不会真正稳定下来——如果电路运行足够长的时间，从 TAP 设备发出的信号最终会减少到无法测量的程度。另一种设想方式该负反馈环路呈螺旋状，信号强度以螺旋状表示，并用图表显示输出信号随时间的推移而减少，如图 8.2所示。

The result, as you can see from the output signal on the right side of the illustration, is a signal that is constantly decreasing in strength. It’s tempting to think this situation will stabilize, but it never really will—leave the circuit running long enough, and the signal exiting the TAP device will eventually be reduced to the point that it cannot be measured. An alternative way to envision this negative feedback loop is as a spiral, with the strength of the signal represented as a spiral and a graph to show the decreasing signal on the output over time as shown in Figure 8.2.

图 8.2 负反馈环替代概念

Figure 8.2 Negative Feedback Loop Alternate Conceptions

图 8.2左侧的螺旋图将电路的输入和输出显示为一组周期，每个周期都接近螺旋中间的零点，或者信号变得不可测量的点。图 8.2右侧的曲线版本显示了信号强度不断降低的图表，并标记了相同的周期以供参考。

The spiral chart, on the left in Figure 8.2, shows the input and output of the circuit as a set of cycles, with each cycle moving closer to the null point in the middle of the spiral, or the point where the signal becomes unmeasurable. The curve version, on the right in Figure 8.2, shows the constantly reducing signal strength against a graph, with the same cycles marked out for reference.

为了说明正反馈的外观，请反转图 8.1中所示的 TAP 设备的操作；图 8.3说明了结果。

To illustrate a positive feedback look, reverse the action of the TAP device shown in Figure 8.1; Figure 8.3 illustrates the result.

图 8.3 正反馈环路

Figure 8.3 A Positive Feedback Loop

在这种情况下，TAP 设备配置为使信号与原始信号同相，并稍微降低其强度（更准确地说是幅度）。该插图中发生的情况是这样的：

In this case, the TAP device is configured to leave the signal in phase with the original and reduce its strength (amplitude in more precise terms) a bit. What happens in this illustration is this:

1.标记为 A 的波形部分穿过 ADD 器件。

1. The section of the wave marked A passed through the ADD device.

2.起初，第二个输入上没有信号，因此波没有变化地通过 ADD 器件。

2. At first, there is no signal on the second input, so the wave passes through the ADD device without change.

3.波继续前进并进入 TAP 设备。

3. The wave continues on and passes into the TAP device.

4.在 TAP 设备处，制作波的小副本，并且该信号的小副本沿着路径传回 ADD 设备的第二个输入。

4. At the TAP device, a small replica of the wave is made, and this little replica of the signal is passed back along the path to the second input of the ADD device.

5.从TAP设备反馈的该信号由ADD设备添加到原始信号中。

5. This signal, being fed back from the TAP device, is added to the original signal by the ADD device.

6.由于该信号未反转，因此两个波的相加具有增加 ADD 设备的输出强度的效果。

6. Because this signal is not inverted, the addition of the two waves has the effect of increasing the strength of the output of the ADD device.

7.这个过程很像取一个大数，乘以某个非常小的数（例如 0.01），然后将其加回到原始数。

7. This process is much like taking a large number, multiplying it by some very small number (say 0.01), and adding it back to the original number.

8.这会导致 ADD 设备的输出电平增加。

8. This has the effect of causing the output level of the ADD device to increase.

9.然后，该增加的信号被反馈到 TAP 设备，TAP 设备采用相同比例的接收信号并将其反馈到 ADD 设备。

9. This increased signal is then fed back into the TAP device, which takes the same proportion of the received signal and feeding it back to the ADD device.

再次强调，有时以不同的方式说明同一概念很有用，如图8.4所示。

Once again, it is sometimes useful to illustrate the same concept in a different way, as shown in Figure 8.4.

图 8.4 正反馈环路的替代图示

Figure 8.4 An Alternate Illustration of a Positive Feedback Loop

看看这两个插图（图 8.3和8.4），正反馈循环为何危险就显而易见了。第一的，除非有什么东西限制了结果信号的幅度，否则循环就没有尽头。事实上，大多数正反馈循环确实发生在带有某种类型限制器的系统中。大多数时候，限制器是系统本身的一些物理属性。举个例子，考虑一下音频系统中的反馈——当麦克风距离扬声器太近时，您有时会听到很大的刺耳噪音。这种类型的反馈是由扩音器和扬声器系统中的任何噪声（总会有一些）引起的，这些噪音被麦克风（上图中的 TAP）拾取，然后反馈到扩音系统（上图中的 ADD）），产生更高音量的噪音，然后被麦克风拾取为更大的噪音，从而放大到更高的水平，然后由扬声器播放。

Looking at these two illustrations (Figures 8.3 and 8.4), it should be obvious why positive feedback loops are dangerous. First, unless there is something that limits the amplitude of the resulting signal, there is no end to the loop. In fact, most positive feedback loops do occur in systems with a limiter of some type; most of the time the limiter is some physical property of the system itself. As an example, consider feedback in an audio system—that loud screeching noise you hear sometimes when a microphone is placed too close to a speaker. Feedback of this type is caused by any noise in the amplification and speaker system (and there will always be some) being picked up by a microphone (TAP in the diagrams above), then fed back into the amplification system (ADD in the illustrations above), to create a higher volume noise, which is then picked up by the microphone as a louder noise, and hence amplified to a higher level, and then played by the speakers. The limiting factor in the audio example is the upper limit on the speaker’s volume, combined with the physical properties of the microphone.

其次，一旦系统达到限制器，它将永久停留在那里。TAP/ADD 反馈的作用会立即克服变化率或强度的降低，该反馈会添加所需的任何级别的信号（随着时间的推移）以使系统回到限制点。这在控制系统和电子领域称为饱和。

Second, once the system reaches the limiter, it will stay there permanently. And reduction in the rate or strength of change is immediately overcome by the action of the TAP/ADD feedback, which adds in whatever level of signal is needed (over time) to bring the system back to the limiting point. This is called saturation in the field of control systems and electronics.

笔记

Note

这里的插图表明，正反馈循环总是会导致输出水平的增加，因为在处理中添加了输入和输出的某些部分。然而，许多正反馈循环只是实现稳定性，因此被称为自强化或稳定反馈循环。在这种情况下，循环不会增加输出，而只是将输出稳定在较高的水平。结果是输入到输出的不断放大，而不是不断增加的输出。为了简化讨论，本文中这些情况仍然被视为正反馈循环。

The illustrations here indicate that a positive feedback loop will always result in an increasing level of output, as the input and some part of the output are added in the processing. However, many positive feedback loops simply achieve stability, and hence are called self-reinforcing, or stable, feedback loops. In this case, the loop doesn’t add to the point of increasing output, but simply keeps the output stable at an elevated level. The result is a constant amplification of the input to the output, rather than an ever increasing output. To simplify the discussion, these cases are still treated as positive feedback loops in this text.

正反馈环路和负反馈环路之间的区别在于这些插图中反馈到 ADD 中的数字的符号。如果 ADD 被馈送负数，则反馈回路将为负。如果输入正数，则反馈循环将为正。反馈的陡度（即反馈影响输出信号的速率）受到 TAP 处发出并反馈至 ADD 的原始信号百分比的影响。百分比越高，输出增加或减少的速度越快。

The difference between a positive feedback loop and a negative feedback loop is the sign of the number fed back into the ADD in these illustrations. If the ADD is fed a negative number, the feedback loop will be negative. If it’s fed a positive number, the feedback loop will be positive. The steepness of the feedback, the rate at which it impacts the output signal, is impacted by the percentage of the original signal that’s taken off at the TAP and fed back to the ADD. The higher the percentage, the greater the speed at which the output increases or decreases.

网络工程中的正反馈循环

Positive Feedback Loops in Network Engineering

反馈循环（特别是正反馈循环）与网络工程有何关系？回到第 2 章“复杂性的组成部分”中描述的网络复杂性的原始模型，反馈循环与复杂性模型的所有三个部分相互作用：状态、速度和表面。通过一些示例将是理解反馈循环如何与复杂性相关的最佳方式；本节将考虑数据包复制环路、重新分配环路和链路抖动控制平面故障环路。

How do feedback loops—positive feedback loops in particular—relate to network engineering? Returning to the original model of network complexity described in Chapter 2, “Components of Complexity,” feedback loops interact with all three pieces of the complexity model: state, speed, and surface. Working through some examples is going to be the best way to understand how feedback loops relate to complexity; this section will consider a packet duplication loop, a redistribution loop, and a link flap control plane failure loop.

数据包循环

在任何给定网络中，数据包循环比最初想象的更为常见，但它们并不总是形成反馈循环，因此并不总是与网络故障相关。图 8.5说明了典型的数据包循环。

Packet loops are more common than might be initially supposed in any given network—but they don’t always form feedback loops, and hence aren’t always related to a network failure. Figure 8.5 illustrates a typical packet loop.

图 8.5 数据包循环示例

Figure 8.5 Packet Loop Examples

图 8.5的左半部分说明了一个不发生数据包复制的循环，因此不存在立即明显的正反馈循环。主机A发送的报文经过RouterB转发到RouterC，然后再转发到RouterD，然后返回RouterB，再转发到RouterC。但是，存在一个自续环路，或者说足够多的环路。正反馈以维持网络中链路上增加的流量水平。如果主机 A 传输足够的流量，则这种自维持环路仍然会导致流量达到环路的限制器或饱和点。

The left half of Figure 8.5 illustrates a loop in which packet replication does not take place—so there is no immediately obvious positive feedback loop. Packets transmitted by Host A are forwarded by Router B to Router C, which are then forwarded to Router D, then back to Router B, where they are again forwarded to Router C. There is, however, a self-sustaining loop, or enough positive feedback to sustain the increased traffic levels across the links in the network. This self-sustaining loop can still cause the amount of traffic that can reach the limiter, or saturation point, for the loop, if Host A is transmitting enough traffic.

要理解其中的原因，请考虑防止流量永远沿着此环路传递的唯一方法是在数据包本身中设置最大跳数（生存时间）。如果生存时间设置得非常高，以允许宽直径网络（具有很多跳数的网络），则主机 A 传输的每个数据包都会增加网络上的负载，而较旧的数据包仍在倒计时为了活着。如果生存时间设置为 16，则主机 A 最多可以消耗从自身到路由器 B 以及其他路由器的链路上可用带宽的 16 倍。网络中的链接。如果主机 A → 路由器 B 链路为 1g，则主机 A 在 [B,C]、[C,D] 和 [D,B] 链路上最多可消耗 16g（合计，或每条链路上 16g 的三分之一）三个链接）。

To understand why, consider that the only way to prevent traffic from passing along this loop eternally is to put a maximum number of hops in the packet itself—a time to live. If the time to live is set very high, to allow for a wide diameter network (a network with a lot of hops), then each packet Host A transmits will increase the load on the network while older packets are still counting down to their time to live. If the time to live is set to 16, Host A can consume up to 16 times the bandwidth available along the link from itself to Router B along other links in the network. If the Host A → Router B link is 1g, then Host A can consume up to 16g across the [B,C], [C,D], and [D,B] links (combined, or one third of 16g on each of the three links).

图 8.5的右侧说明了具有正反馈特性的转发环路。这里，路由器 E 发送到路由器 F 的数据包由路由器 G 转发到路由器 H 和 K，每个路由器都会复制该数据包并将其发送回路由器 F。单个数据包每次穿过环路时都会加倍；那么，第一轮，主机E发送的单个报文就变成了2个，第二轮就变成了4个，第三轮就变成了8个，以此类推。显然，这只是多播或广播流量情况下的问题；单播流量通常不会在路由器 F 上复制。

The right side of Figure 8.5 illustrates a forwarding loop with positive feedback characteristics. Here a packet transmitted by Router E to Router F is forwarded by Router G to both Routers H and K, each of which replicates the packet and transmits it back to Router F. Each time the single packet traverses the loop, it doubles; in the first round, then, the single packet transmitted by Host E becomes two, in the second round it becomes four, in the third round it becomes eight, and so on. It should also be obvious this is only a problem in the case of multicast or broadcast traffic; unicast traffic wouldn’t normally be duplicated at Router F.

这些转发循环的限制器是什么？限制因素包括：

What would the limiter be for either of these forwarding loops? Limiters would include:

• 转发路径中任何设备转发流量的速度。

• The speed at which any device in the forwarding path can forward the traffic.

• 连接设备的链路的带宽。

• The bandwidth of the links connecting the devices.

• 传输到转发环路的数据包的生存时间。这通常不会成为太大的限制因素，因为任何生存时间通常都设置得足够高，以确保通过大直径网络的传输，而这几乎总是太高而无法防止转发环路造成的附带损害。

• The time to live of the packets being transmitted into the forwarding loop. This generally isn’t going to be much of a limiter, as any time to live is normally set high enough to ensure transport across a large diameter network, which is almost always too high to prevent collateral damage from a forwarding loop.

• 控制平面状态在转发环路所经过的链路上传递的可行性。

• The viability of the control plane state passing over the links through which the forwarding loop passes.

虽然其中大多数都是相当明显的，但最后一个限制因素可能不是——控制平面的可行性到底意味着什么？如果环路中涉及的链路上的流量变得足够大而导致数据包丢失，则控制平面将无法维持状态。当控制平面状态失败时，形成链路的可达性信息将（可能）从表中删除，从而导致循环“展开”。如果环路稳定，当控制平面重新学习环路中链路的可达性信息时，环路将重新形成。

While most of these are fairly obvious, the last limiter might not be—what precisely does the viability of the control plane mean? If the traffic along the links involved in the loop becomes large enough to cause packet drops, the control plane will not be able to maintain state. When the control plane state fails, the reachability information that formed the link will (likely) be removed from the table, causing the loop to “unwind.” If the loop is stable, it will be re-formed when the control plane relearns reachability information across the links in the loop.

控制平面中的数据包循环

网络中如何形成导致反馈循环的转发循环？一般来说，问题始于控制平面的某个地方。一般来说，当网络的实际状况不佳时，控制平面就会形成转发环路。与控制平面的网络视图不匹配。一些示例可能有助于理解这种情况如何以及何时发生。

How do forwarding loops that lead to feedback loops form in a network? Generally the problem begins someplace in the control plane. Generally speaking, control planes will form forwarding loops when the actual state of the network doesn’t match the control plane’s view of the network. Some examples might be helpful in understand how and when this happens.

相互再分配

图 8.6说明了两个路由协议之间相互重新分配的网络。

Figure 8.6 illustrates a network with mutual redistribution between two routing protocols.

图 8.6 重新分配控制平面循环

Figure 8.6 Redistribution Control Plane Loop

在此网络中，路由器 A 将 2001:db8:0:1::/64 重新分配到 OSPF v3 中，发送给路由器 B 和 D，成本为 100。为了简化说明，取环路的一侧；路由器 B 将此目标重新分配到 EIGRP 中，度量值为 1000。然后，路由器 D 将 2001:db8:0:1::/64 作为 EIGRP 外部路由选取，并重新分配回 OSPF，成本为 10。路由器 D然后沿着RouterA、B、D共享的广播链路将该路由发布给RouterB；该路由作为外部 OSPF 路由到达路由器 B，成本为 10。因为通过路由器 D 的成本（10 加上 [B,D] 链路的成本）小于通过路由器 A 的路由成本（100 加） [B,D] 链路的成本 — 路由器 B 将选择通过路由器 D 到达此目的地的路径。每次在两个协议之间重新分配目的地时，这种类型的路由循环可能会因成本增加而自行超时，但一旦第一个重新分配周期完成，它就会很快重建。这种类型的路由循环也可以保持稳定，具体取决于选择重新分配度量的方式。

In this network, Router A is redistributing 2001:db8:0:1::/64 into OSPF v3 toward Routers B and D with a cost of 100. To simplify the explanation, take one side of the loop; Router B redistributes this destination into EIGRP with a metric of 1000. 2001:db8:0:1::/64 is then picked up as an EIGRP external route by Router D, and redistributed back into OSPF with a cost of 10. Router D then advertises this route back to Router B along the broadcast link shared by Routers A, B, and D; the route reaches Router B as an external OSPF route with a cost of 10. Because the cost through Router D—10 plus the cost of the [B,D] link—is less than the cost of the route through Router A—100 plus the cost of the [B,D] link—Router B will choose the path through Router D to reach this destination. This type of routing loop can time itself out through the increasing costs each time the destination is redistributed between the two protocols, but it will quickly be rebuilt once the first cycle of redistribution is completed. This type of routing loop can also remain stable, depending on the way the redistribution metrics are chosen.

笔记

Note

这样做的原因是原始路由被重新分配到 OSPF 中，而不是简单地通告到 OSPF 中。2001:db8:0:1::/64 必须是两个入口点的外部路由才能创建循环；否则，路由器 B 和路由器 D 将拥有 1 条内部 OSPF 路由和 1 条外部 OSPF 路由，并且它们将始终优先选择内部路由而不是外部路由，从而打破环路。需要多个重新分配点才能导致这种类型的循环。

The reason this works is the original route is redistributed into OSPF, rather than simply being advertised into OSPF. 2001:db8:0:1::/64 must be an external route at both points of entry to create the loop; otherwise Routers B and D will have one internal OSPF route and one external OSPF route, and they will always prefer the internal route over the external, breaking the loop. Multiple points of redistribution are required to cause this type of loop.

人们很容易认为这里的问题是从一个路由域传递的信息被泄漏或重新分发到另一个路由域，然后又被重新分发回来——相互重新分发实际上是问题的根源。但是，虽然消除相互重分配或阻止重新分配的路由再次重分配（通过过滤器、标签、社区或其他机制）可以解决问题，但相互重分配并不是根本原因。

It’s tempting to say the problem here is that information passing from one routing domain is being leaked, or redistributed, into another routing domain, and then it’s being redistributed back again—that the mutual redistribution is actually the root of the problem. But while removing the mutual redistribution, or blocking redistributed routes from being redistributed again (though filters, tags, communities, or other mechanisms) will resolve the problem, mutual redistribution is not the root cause.

根本原因实际上是在重新分配过程本身中删除了有关网络状态的信息。回到路由基础知识，您可能还记得路由协议通过检查指标来确定到目的地的特定路径是否是环路。由于无法直接比较任何两个协议使用的度量，因此分配给重分配路由的度量必须以某种方式简单地配置或计算 - 无论如何完成此配置或计算，有关网络状态的信息（成本）在这种情况下到目的地的路径）在重新分配路由时可能会丢失。

The root cause is actually the removal of information about the state of the network in the redistribution process itself. Reaching back to routing fundamentals, you might recall that routing protocols determine whether a particular path to a destination is a loop by examining the metrics. Because the metrics of any two protocols use cannot be directly compared, the metric assigned to a redistributed route must simply be configured or calculated in some way—no matter how this configuration or calculation is done, information about the state of the network (the cost of the path to the destination in this case) will likely be lost when redistributing routes.

因此，当在两个不同的路由协议之间重新分配路由信息时，网络的实际状态与控制平面认为的网络状态之间会发生不匹配。每当发生这种不匹配时，就有可能出现路由环路，从而导致网络中出现（可能是永久性的）转发环路。

Hence, when routing information is redistributed between two different routing protocols, a mismatch between the actual state of the network and what the control plane believes to be the state of the network occurs. Any time such a mismatch occurs, there is the possibility of a routing loop, which then causes a (potentially permanent) forwarding loop in the network.

微循环

第 7 章“协议复杂性”花费了大量时间研究微循环和各种解决方案，特别是在增加协议复杂性方面，试图达到图灵曲线的“拐角”（参见第 1 章“定义复杂性”）。使用图 8.7在转发循环的上下文中快速回顾一下这里将会很有用。

Chapter 7, “Protocol Complexity,” spent a good deal of time examining microloops and various solutions, specifically in terms of increasing protocol complexity in an attempt to reach the “corner” of the Turing Curve (see Chapter 1, “Defining Complexity”). A quick revisit here in the context of forwarding loops will be useful, using Figure 8.7.

图 8.7 控制平面中的微循环

Figure 8.7 A Microloop in the Control Plane

假定该网络正在运行链路状态协议，并且路由器 D 到 2001:db8:0:1::/64 的最佳路径是通过路由器 C，[B,C] 处的故障将导致路由器 C 和路由器之间形成环路。 D 在Router C 收敛之后、Router D 收敛之前的时间内。第 7 章“协议复杂性”讨论了该问题的多种解决方案，但根本原因到底是什么？

Given this network is running a link state protocol, and Router D’s best path to 2001:db8:0:1::/64 is through Router C, a failure at [B,C] will cause a loop to form between Routers C and D during the time after Router C has converged and before Router D has converged. Chapter 7, “Protocol Complexity,” discussed a number of solutions for this problem—but what, really, is the root cause?

虽然最明显的问题是两个路由器应该同时重新计算，但实际的根本原因是现有的网络拓扑与路由器 D 认为的网络拓扑不匹配。这强化了这样的观点：只要存在的拓扑与控制平面的拓扑视图之间存在不匹配，就会发生不好的事情。

While the most obvious problem is the two routers should recalculate at the same time, the actual root cause is the mismatch between the network topology as it exists and the network topology as Router D believes it to be. This reinforces the point that any time there is a mismatch between the topology as it exists and the control plane’s view of the topology, bad things happen.

从这个角度来看，问题为何如此难以解决就显而易见了。CAP 定理指出，您可以设计一个一致、可用且具有分区容错性的数据库（也许应该是 CAT 定理，以纪念那种难以解释的生物——猫而命名？）——您必须在三个中选择两个。如果简单地将控制平面视为分布式实时数据库，那么路由协议必须放弃一些东西——应该是哪一个？

Seen from this angle, it becomes obvious why the problem is so difficult to solve. The CAP theorem states that you can design a database that is consistent, available, and exhibits tolerance for partitions (perhaps it should be the CAT theorem, named in honor of that inexplicable creature, the cat?)—you must choose two of the three. If the control plane is simply treated as a distributed real time database, the routing protocol must give up something—which one should it be?

对于分区的容忍度当然不是分布式路由协议可以放弃的；这将破坏其分布式本质的本质。可访问性是另一个不容置疑的领域。参与控制平面的每个设备都无法始终访问路由协议数据库，对于数据包转发任务来说确实没有多大用处。那么，一致性是分布式控制平面中必须让步的一点——正如第 7 章对微循环问题的探索所示，不存在任何问题。解决来自网络不一致视图的问题的简单方法（事实上，根据 CAP 定理，无论问题有多么复杂，都没有办法真正完全解决问题）。

Tolerance for partitioning is certainly not something a distributed routing protocol can give up; that would destroy the very essence of its distributed nature. Accessibility is another area that simply isn’t in doubt; a routing protocol database that isn’t accessible by every device participating in the control plane all the time really isn’t much use for packet forwarding duty. Consistency, then, is the point that must give way in a distributed control plane—and as Chapter 7’s exploration of the problem of microloops illustrates, there’s no simple way to resolve the problems that come from an inconsistent view of the network (in fact, according to CAP theorem, there’s no way to actually resolve the problem entirely, no matter how much complexity is thrown at the problem).

当然，还可以做出另一组选择——控制平面数据库可以集中化，从 CAP 定理中删除“P”，从而使“A”和“C”在理论上成为可能。然而，在现实生活中并不总是能解决这个问题，正如您将在第 10 章“可编程网络复杂性”中发现的那样。

Another set of choices can be made, of course—the control plane database can be centralized, removing the “P” out of the CAP theorem, and hence making the “A” and “C” theoretically possible. It doesn’t always work out this in real life, however, as you’ll discover in Chapter 10, “Programmable Network Complexity.”

速度、状态和表面：网络控制平面的稳定性

Speed, State, and Surface: Stability in the Network Control Plane

了解了背景中的反馈循环后，是时候回到复杂的世界了。本节将从控制平面循环的另一个示例开始，然后继续讨论与网络工程相关的速度、状态和复杂性表面。

With this understanding of feedback loops in the background, it’s time to move back into the world of complexity. This section will begin with another example of a control plane loop, and then continue with a discussion of the speed, state, and surface of complexity in relation to network engineering.

生成树故障案例研究

生成树因级联控制平面故障而闻名（也许“臭名昭著”是一个更好的术语）。图 8.8说明了一个小型网络，可以通过该网络追踪此类故障。

Spanning Tree is famous—perhaps infamous is a better term—for cascading control plane failures. Figure 8.8 illustrates a small network over which it is possible to trace such a failure.

图 8.8 生成树故障示例

Figure 8.8 Spanning Tree Failure Example

该网络以交换机 A 作为根桥开始，路径 [B,D] 被生成树协议阻止。假设有两件事同时发生：

This network begins with Switch A as the root bridge, and the path [B,D] blocked by the Spanning Tree Protocol. Assume that two things occur at the same time:

• 超大单向流正在通过交换网络，使用大量可用带宽，从交换机 A 通过交换机 B 和 C，到达交换机 E（以及交换机 E 之外的某个位置）。

• An outsized unidirectional flow is passing through the switched network, using a large amount of the available bandwidth, from Switch A through Switches B and C, to Switch E (and someplace beyond Switch E).

• 交换机 B 上的某些进程出现异常，导致桥接协议数据单元 (BPDU) 进程无法发送常规 hello 数据包。

• Some process on Switch B misbehaves, causing the Bridge Protocol Data Unit (BPDU) process to fail in sending regular hello packets.

如果生成树进程无法在交换机 B 处发送常规 BPDU，交换机 C 和 D 将开始选择新的根桥，从而确定通过网络的最短路径集，而不参考交换机 A 的当前状态作为根桥或交换机 B 的存在。这将导致 [B,D] 链路的流量畅通，因此：

If the spanning tree process fails to send regular BPDUs at Switch B, Switches C and D will begin to elect a new root bridge, and hence to determine the shortest set of paths through the network without reference to the current state of Switch A as the root bridge, or the existence of Switch B. This will result in the [B,D] link being unblocked for traffic, so:

• 源自A 的大流量将通过广播链路[B,C,D] 传输。

• The large flow originating at A will be transmitted across the broadcast link [B,C,D].

• 大流量将由交换机C 和D 传输到广播链路[C,D,E]。

• The large flow will be transmitted by both Switches C and D onto the broadcast link [C,D,E].

• 交换机D 将在其[C,D,E] 接口接收相同的大流量，并将其重新传输回[B,C,D] 广播链路，从而建立转发环路。

• Switch D will receive the same large flow at its [C,D,E] interface and retransmit it back onto the [B,C,D] broadcast link, setting up the forwarding loop.

一旦转发环路启动，唯一的限制因素将是网络中链路、接口和转发设备的饱和点。随着线路上流量的增加，生成树所依赖的沿网络拓扑形成最短路径树的 BPDU 将被丢弃，从而导致网络进一步分段为独立的交换机，每个交换机都选择自己的根桥。一旦网络达到这一点，除了一台（或多台）交换机发生故障之外，没有其他方法可以恢复，从而导致流量停止转发，并使生成树有机会通过网络重新建立无环路路径。

Once the forwarding loop is started, the only limiting factor will be the saturation point for the links, interfaces, and forwarding devices in the network. As the traffic on the wire builds, the BPDUs that Spanning Tree counts on to form a shortest path tree along the network topology will be dropped, causing the network to further fragment into independent switches each electing their own root bridge. Once the network reaches this point there is no way to recover other than the failure of one (or more) of the switches, causing the traffic to stop being forwarded, and giving Spanning Tree the chance to re-establish loop free paths through the network.

乘坐跷跷板

到目前为止给出的例子中有几点需要考虑；这里将使用状态/速度/表面模型将这些点转化为复杂性术语。

There are several points to be considered in the examples given thus far; the state/speed/surface model will be used here to put these points into complexity terms.

状态：在每种情况下，控制平面状态在启动或维持反馈循环中发挥着作用。问题是缺乏有关网络拓扑真实状态的信息。每当控制平面所认为的网络拓扑与现实世界中存在的实际网络拓扑不匹配时，就会出现一些转发问题。上面的示例使用数据包循环通过数据包丢失（生成树故障）来删除拓扑信息，并删除描述实际网络拓扑的控制平面状态（重新分配）来说明状态与现实之间的不匹配。

State: In each of these cases, control plane state plays a role in starting or maintaining the feedback loop. The problem is a lack of information about the true state of the network topology. Any time there is a mismatch between what the control plane believes about the network topology, and the actual network topology as it exists in the real world, there will be some forwarding issue. The examples above used a packet loop to remove topology information through packet loss (spanning tree failure), and the removal of control plane state describing the actual network topology (redistribution) to illustrate this mismatch between state and reality.

速度：控制平面中的信息丢失，导致网络与控制平面的网络视图之间不匹配，是此处给出的每个示例的主要原因。然而，重要的是要注意速度在每种情况下所扮演的角色。在微循环的情况下，控制平面的速度不够快，无法跟上网络拓扑的变化。然而，加速控制平面只会导致其他复杂性问题浮出水面——复杂性的世界里没有免费的午餐（或者更确切地说，你不能像设置一个计时器那样轻松地击败复杂性恶魔）让事情进展得更快）。

Speed: The loss of information in the control plane, resulting in a mismatch between the network and the control plane’s view of the network, is the primary cause for each of the examples given here. However, it is important to note the role speed plays in each situation, as well. In the case of microloops, the control plane isn’t fast enough to keep up with changes in the network topology. Speeding the control plane up, however, simply causes other complexity issues to surface—there is no such thing as a free lunch in the world of complexity (or, rather, you can’t beat the complexity demon as easily as just setting a timer to make things go faster).

另一种看待这个问题的方法是跷跷板：网络对拓扑变化的反应越快，就越有可能发现控制平面形成正反馈循环的情况。速度好，但速度也不好。如果没有考虑到当重量转移并且你的一侧开始向地面移动时会发生什么，就很容易推到跷跷板的一侧。当你“看到”而不是“看到”时，你会面对什么？底部有一个令人讨厌的肿块，或者更糟糕的东西？

An alternative way to look at this is as a see-saw: the fast the network reacts to topology changes, the more likely you are to find situations where the control plane forms a positive feedback loop. Speed is good, but speed is also bad. It’s far too easy to push to one side of the see-saw without thinking through what’s going to happen when the weight shifts, and your side starts heading toward the ground. What is it you’ll face when you “saw” rather than “see”? A nasty bump on the bottom, or something worse?

表面：刚才讨论的两个例子说明了连接两个不同系统的宽而深的表面的问题。在重分配的例子中，两种路由协议之间的表面可以说不够深入。由于协议之间没有完全交换信息，因此交互界面效率低下。此外，这种情况下的交互面太宽了；重新分发不需要在网络中超过最少数量的位置进行。减少重新分配发生的点可能会降低通过网络的流量效率，但这是针对控制平面状态的常见权衡。

Surface: Two of the examples just discussed illustrate the problem with broad, deep surfaces connecting two different systems. In the redistribution example, the surface between the two routing protocols can be said not to be deep enough. Because information isn’t fully exchanged between the protocols, the interaction surface is inefficient. Further, the interaction surface in this case is too broad; redistribution doesn’t need to take place at more than a minimal number of places in the network. Reducing the points at which redistribution does take place can potentially reduce the efficiency of traffic flow through the network, but this is a common tradeoff against control plane state.

在生成树故障的情况下，所看到的交互面是控制平面和流经网络的流量或数据平面之间。这通常不是网络工程师考虑的交互表面，但由于控制平面与数据平面位于相同的链路上（控制平面位于 band 中），因此需要考虑一个明确的交互表面。

In the case of the spanning tree failure, the interaction surface in view is between the control plane and the traffic flowing through the network, or the data plane. This isn’t normally an interaction surface network engineers think about, but because the control plane rides on the same links as the data plane (the control plane is in band), there is a definite interaction surface that needs to be considered.

第三个例子，微循环，以另一种方式说明了表面问题。控制平面可以说是一个系统，而网络拓扑可以说是一个单独的系统。因此，控制平面对拓扑状态的检测是一个交互面。在这种情况下，交互表面可以说太浅了，因为控制平面不能总是像实际发生的拓扑变化一样快地做出反应。这是看待上面段落中描述为状态和速度问题的问题的另一种方式。

The third example, microloops, illustrates the surface problem in another way. The control plane can be said to be one system, while the network topology can be said to be a separate system. The detection of the topology state by the control plane is therefore an interaction surface. In this case, the interaction surface can be said to be too shallow, in that the control plane cannot always react to changes in the topology as quickly as those changes actually occur. This is another way of looking at the problem described as a state and speed issue in the paragraphs above.

命运共同体

Shared Fate

系统之间或系统内部的反馈循环很有趣，但一旦您知道自己在寻找什么，它们通常很容易被发现。然而，在任何给定的网络中，通常很难看到共同的命运——最常见的原因是共同的命运情况被故意隐藏在旨在降低网络表面复杂性的抽象层之下。什么是共命运问题？它如何影响现实世界中的网络故障？理解此类问题的最佳方法是通过示例。一旦考虑了这些例子，共同的命运问题将与本书中使用的复杂性模型（速度、状态和表面）相关联。

Feedback loops between systems or within a system are interesting, but they are often easy to spot once you know what you’re looking for. Shared fate, however, is often very difficult to see in any given network—most often because shared fate situations are intentionally buried under layers of abstraction designed to reduce the apparent complexity of the network. What is a shared fate problem, and how does it impact network failure in the real world? The best way to understand this type of problem is through examples. Once these examples have been considered, shared fate problems will be related back into the complexity model used throughout this book—speed, state, and surface.

虚拟电路

Virtual Circuits

虚拟电路并不新鲜——事实上，使用标记的数据包头将几个不同的“电路”放置到一条物理线路上几乎从一开始就在网络中很常见。从多路复用 T1 开始，通过帧中继工作，以及放置在以太网帧上的更现代的 802.1Q 和 802.1ad 标头，将单个以太网链路分解为多个虚拟拓扑，虚拟化已成为数据链路中的规则，而不是例外。协议。如今，工程师可以选择 VXLAN、MPLS 和许多其他技术来虚拟化其链路。

Virtual circuits aren’t new—in fact, the use of tagged packet header to put several different “circuits” onto a single physical wire have been common from almost the very beginning in networks. Starting with multiplexed T1’s, working through Frame Relay, and the more modern 802.1Q and 802.1ad headers placed on an Ethernet frame to break a single Ethernet link into multiple virtual topologies, virtualization has been the rule, rather than the exception, in data link protocols. Today, engineers can choose from VXLAN, MPLS, and many other technologies to virtualize their links.

虚拟化提供了许多好处，例如：

Virtualization provides many benefits, such as:

• 能够对另一个虚拟拓扑隐藏一个虚拟拓扑；一种信息隐藏形式，可减少覆盖控制平面中携带的状态，以及控制平面对拓扑变化必须做出反应的速度。

• The capability to hide one virtual topology from another; a form of information hiding that reduces the state carried in the overlay control plane, and the speed at which the control plane must react to changes in the topology.

• 将物理拓扑抽象为多个逻辑拓扑的能力，每个逻辑拓扑都有自己的特性，例如每跳行为。

• The capability to abstract the physical topology into multiple logical topologies, each with their own characteristics, such as per hop behaviors.

• 通过比最短路径更长的路径传输流量（以增加特定路径的长度）以满足特定业务或运营目标的能力。

• The capability to carry traffic across a longer than shortest path (to increase the stretch of a specific path) to meet specific business or operational goals.

然而，虚拟化也有一个缺点——SRLG。图 8.9说明了共享风险链接组。

There is a downside to virtualization, however—the SRLG. Figure 8.9 illustrates shared risk link groups.

图 8.9 共享风险链接组

Figure 8.9 Shared Risk Link Groups

假设如下：

Assume the following:

• 路由器A 和B 都是提供商X 的分界（或切换）点，该提供商正在销售两个城市之间的高速链路。

• Routers A and B are both demarcation (or handoff) points for Provider X, which is selling a high-speed link between two cities.

• 路由器C 和D 都是提供商Y 的分界（或切换）点，该提供商正在销售两个城市之间的高速链路。

• Routers C and D are both demarcation (or handoff) points for Provider Y, which is selling a high-speed link between two cities.

客户从提供商 X 购买虚拟电路，并从提供商 Y 购买备份链路，以连接两个提供商互连的两个城市中的设施。然后，反铲操作员开始对 E 路段的网络进行褪色（也许他们正在清理路边某处的排水沟）。

A customer purchases a virtual circuit from Provider X, and a backup link from Provider Y, to connect their facilities in the two cities that both providers interconnect. A backhoe operator then proceeds to fade the network at Link E (perhaps they are cleaning out a drainage ditch someplace alongside a road).

这将导致两个虚拟电路都失败，因为它们共享单个物理链路。

This will cause both of the virtual circuits to fail, as they share a single physical link.

客户怎样才能避免这种情况呢？两家提供商都不太可能准确解释其虚拟电路是如何逐跳配置的，这将提供可能对提供商的业务产生负面影响的竞争信息。从客户的角度来看，单个链路 E 的虚拟化创建了一个不可见的 SRLG，因为抽象的魔力，客户对此无能为力，只能使用单个提供商并要求这种类型的服务。网络中不会出现这种情况。

How can the customer avoid this situation? Neither provider is likely to explain exactly how their virtual circuit is provisioned, hop-by-hop—this would be giving out competitive information that could negatively impact the provider’s business. From the customer’s perspective, the virtualization of the single link, E, has created an SRLG that is not visible because of the magic of abstraction—and there’s little the customer can do about the situation other than use a single provider and demand this type of situation doesn’t arise in the network.

虚拟化抽象可以说是泄漏；较低逻辑层的状态以共享风险组的形式泄漏到较高逻辑层，其中单个故障会转化为多个不同的中断。

The virtualization abstraction can be said to leak; state at a lower logical layers leaks into higher logical layers in the form of shared risk groups, where a single failure translates into a number of different outages.

TCP 同步是一个共同的命运问题

TCP Synchronization as a Shared Fate Problem

然而，共同命运问题不仅仅存在于虚拟化链路之上；还存在于虚拟化链路之上。任何时候使用虚拟化，都有可能出现命运共同体的情况。多个 TCP 流通过网络中一组缓冲区或队列的行为是共享命运问题的意外变体。图 8.10说明了通过网络的多个 TCP 流的行为。

Shared fate problems don’t just ride atop virtualized links, however; any time virtualization is used, there is the possibility of a shared fate situation developing. The behavior of multiple TCP flows passing through a single set of buffers or queues in a network is an unexpected variant of a shared fate problem. Figure 8.10 illustrates the behavior of several TCP flows passing through a network.

图 8.10 TCP 同步

Figure 8.10 TCP Synchronization

在此图中，三台主机 A、B 和 C 正在向只能通过单个 [D,E] 链路访问的另外三台主机发送三个不同的 TCP 流。检查D处的输出队列，有6个数据包等待传输；三个流中各有一个。如果此输出队列只能容纳三个数据包，则放入队列中的三个数据包将被丢弃。如果使用尾部丢弃机制，队列中最新的三个数据包将被丢弃，即P4、P5和P6。

In this diagram, three hosts, A, B, and C, are sending three different TCP streams to three other hosts that are only reachable across the single [D,E] link. Examining the output queue at D, there are six packets waiting to be transmitted; one from each of the three streams. If this output queue can only hold three packets, then three of the packets placed in the queue will be dropped. If a tail drop mechanism is used, the three newest packets in the queue will be dropped, which means P4, P5, and P6.

请注意，这三个数据包代表每个流中的一个数据包，因此三个 TCP 会话中的每一个都将同时进入慢启动模式。无论三个 TCP 会话是否使用相同的计时器来重建更大的窗口，当 [D,E] 链路达到其最大容量的一定百分比时，都会出现相同的情况，从而导致循环重新启动。结果是图右下角的锯齿利用率图表，显然是链路容量的次优使用。有几种方法可以看待这个问题。

Note that these three packets represent one packet from each stream, so each of the three TCP sessions will go into slow start mode at the same time. Whether or not the three TCP sessions are using the same timers to rebuild to a larger window, the same situation will arise when the [D,E] link reaches some percentage of its maximum capacity, causing the cycle to restart. The result is the sawtooth utilization chart on the bottom right of the figure—clearly a suboptimal use of the link capacity. There are several ways to look at this problem.

首先，这可以被建模为抽象泄漏的情况。每个 TCP 会话都将主机到主机的路径视为独占通道，但事实并非如此，该通道与其他 TCP 会话共享。然后，单个链接被抽象为三个链接，每个链接看起来都是独占链接。底层现实在路由器 D 的单个输出队列中泄漏，迫使上层与底层现实进行交互。

First, this can be modeled as a case of a leaky abstraction. Each TCP session sees the path from host to host as an exclusive channel, but it’s not—the channel is being shared with other TCP sessions. The single link, then, is abstracted into three links, each appearing to be an exclusive link. The underlying reality leaks through in the single output queue at Router D, forcing the upper layers to interact with the underlying reality.

第二，这个问题可以被视为相互作用表面的情况——泄漏抽象解释的“另一面”。在本书使用的复杂性理论模型中，速度、状态和表面是关键点。这里的问题是由底层传输系统（[D,E]链路上的物理和数据链路）和覆盖传输系统（IP 和 TCP）之间的交互引起的。在这两个系统的交汇处，有一个它们交互的表面，特别是路由器 D 处的输出队列。那么，在此处使用的网络复杂性模型中，可以通过删除交互表面（使三个链接三个实际链接，而不是一个共享链接），或者通过添加更多深度和复杂性来解决交互的细微差别。

Second, this problem can be seen as a case of interacting surfaces—the “other side” of the leaky abstraction explanation. Within the model of complexity theory used throughout this book, speed, state, and surface are the key points. Here the problem is caused by the interaction between the underlying transport system (the physical and data link on the [D,E] link) and the overlay transport system (IP and TCP). At the intersection of these two systems there is a surface along which they interact—specifically the output queue at Router D. Within the model of network complexity used here, then, this problem can be solved by either removing the interaction surface (making the three links three actual links, rather than one shared link), or by adding more depth and complexity to resolve the nuances of the interaction. Of course there will be tradeoffs in adding this depth to the interaction surface.

加权随机早期检测（WRED）是用于解决此问题的一种机制。通过从队列中随机丢弃数据包，而不是总是丢弃添加到队列中的最后一组数据包，TCP 进入慢启动的影响会分散到不同会话的时间上，从而阻止各个流的原始同步。然而，WRED 可能会对跨此类队列运行的单个流产生副作用，并且通常很难精确调整 WRED 来管理仅持续几秒钟的微小（鼠标）流以及沿着队列的较大、寿命较长（大象）流。相同的链接。将老鼠流和大象流移动到网络中的不同链路上通常很有用，只是为了减轻 TCP 同步和服务质量缓冲区问题的影响。因此，为了解决特定交互表面中的一个问题，复杂性会向外扩散，就像扔进水中的石头所产生的小波一样。好吧，这很哲学，但你明白了。

Weighted Random Early Detection (WRED) is one mechanism used to resolve this problem. By randomly dropping packets off the queue, rather than always dropping the last set of packets added to the queue, the impact of TCP going into slow start is spread across time on different sessions, preventing the original synchronization of the various streams. However, WRED can have side effects on a single stream running across such a queue, and it’s often difficult to precisely tune WRED to manage both tiny (mouse) flows that only last a few seconds alongside larger, long lived (elephant) flows along the same link. It’s often useful to move mouse and elephant flows onto different links in a network just to mitigate the effects of TCP synchronization and quality of service buffer issues. So in an attempt to resolve one problem in a specific interaction surface, complexity ripples outward like the little wavelets expanding from a stone thrown in the water. Okay, that was very philosophical, but you get the point.

最后回到本节的主题，这个问题可以看作是一个命运共同体问题。在这种情况下，您可以将三个 TCP 流建模为跨公共物理基础设施运行的虚拟电路。路由器 D 上的输出队列是所有三个流都必须通过的共享资源，并且该队列的命运由所有三个流共享。

Finally, to return to the theme of this section, this problem can be seen as a shared fate issue. In this case, you can model the three TCP streams as virtual circuits running across a common physical infrastructure. The output queue at Router D is a shared resource through which all three streams must pass, and the fate of that queue is shared by all three flows.

这个具体的例子很有用，因为相对容易将所有三个视角（泄漏抽象、交互表面和共同命运）视为对同一潜在问题进行建模的不同方法。这三个模型中的每一个都可以提出一组不同的解决方案，甚至可以向您指出任何给定解决方案的问题。

This specific example is useful because it’s relatively easy to see all three perspectives—leaky abstractions, interaction surfaces, and shared fate—as different ways to model the same underlying problem. Each of these three models can suggest a different set of solutions, or even point you to problems with any given solution.

最后的想法

A Final Thought

本章涵盖了复杂性与失败之间相互作用的两个广泛领域：反馈循环和共同命运。这两种情况都可以映射到第 1 章“定义复杂性”中首次描述的复杂性模型，并在整本书中使用：

This chapter has covered two broad areas of the interaction between complexity and failure: feedback loops and shared fate. Both of these situations can be mapped to the complexity model first described in Chapter 1, “Defining Complexity,” and used throughout the book:

状态：网络的实际状态与控制平面看到的状态不匹配是控制平面形成微环的最终原因，而通过各种手段去除实际拓扑的状态往往是导致控制平面形成微环的根本原因。在控制平面中形成更永久转发循环的最终原因。在共享命运的情况下，单个共享资源的状态通过协议栈中上层创建的抽象泄漏，从而影响上层协议的操作。

• State: The mismatch between the actual state of the network and the state as viewed by the control plane is the ultimate cause of microloops formed through the control plane, while the removal of state about the actual topology through a variety of means is often the ultimate cause of more permanent forwarding loops formed in the control plane. In the case of shared fate, the state of a single shared resource leaks through the abstraction created by upper layers in the protocol stack to impact the operation of the upper layer protocol.

•速度：网络实际状态的变化速度可能会压垮控制平面，或者发生得足够快以致控制平面无法实时做出反应。实际上，所有控制平面都是接近实时的，而不是实时的，以提供足够的“缓冲”，以防止控制平面在快速变化的情况下发生故障。

• Speed: The speed of change in the actual state of the network can overwhelm the control plane, or happen quickly enough that the control plane cannot react in real time. In reality, all control planes are near real time, rather than real time, to provide enough “buffer” to prevent a control plane failure in the case of rapid changes.

•表面：拓扑和控制平面之间的交互表面在正反馈循环和许多共享命运问题（例如 TCP 同步）中发挥着重要作用。

• Surface: The interaction surface between the topology and the control plane plays a large role in positive feedback loops and in many shared fate problems, such as TCP synchronization.

根本原因分析的思考

Thoughts on Root Cause Analysis

本章讨论了故障的根本原因，但这是工程师需要谨慎对待的概念。复杂的系统，特别是高度冗余和可用的系统，总是处于伪故障模式。任何真正复杂的系统中总会有某个地方出现问题。用于管理和控制故障影响的技术通常足以防止整个系统故障，因此整个系统不会受到影响。作为其结果（或推论），当发生重大或系统性故障时，通常有多个原因。一种常见的情况是，某些伪故障集与拓扑、控制平面或转发平面中的单个转变相结合，充当催化剂，从而导致系统故障。

This chapter has discussed root causes for failures, but this is a concept engineers need to be cautious with. Complex systems, particularly highly redundant and available ones, are always in a pseudo-failure mode. There is always something wrong someplace in any truly complex system. The techniques used to manage and control the impacts of failures are generally sufficient to prevent a total system failure, so the overall system is not impacted. As a result (or corollary) of this, when a major, or systemic, failure occurs, it usually has more than one cause. A common situation is the combination of some set of pseudo-failures with a single shift in the topology, control plane, or forwarding plane that acts as a catalyst, resulting in a systemic failure.

这意味着，在管理大型复杂系统中的系统故障或进行事后分析时，寻找单一根本原因通常会适得其反。相反，事后分析应该做的是尝试找到原始的条件、催化剂、反馈循环以及导致从一次改变到最终失败的交互界面。在复杂的系统中，故障不是单一的事情，它本身就是一个复杂的系统。

This implies that the search for a single root cause is generally counterproductive in managing or doing postmortems of systemic failures in large complex systems. What postmortems should do, instead, is attempt to find the original conditions, the catalyst, and the feedback loops and interaction surfaces that led from the one change to the final failure. In a complex system, a failure isn’t a single thing—it’s a complex system in and of itself.

还需要考虑人性化的一面。罗伯特·库克对人性的一面进行了深思熟虑的表达：

There is a human side to consider, as well. A well thought out expression of this human side is given by Robert Cook:

基于“根本原因”等推理的评估并不反映对失败本质的技术理解，而是反映将结果归咎于特定的、局部的力量或事件的社会、文化需要。。。。这意味着事后事故对人的表现的分析是不准确的。¹

The evaluations based on such reasoning as ‘root cause’ do not reflect a technical understanding of the nature of failure but rather the social, cultural need to blame specific, localized forces or events for outcomes. . . . This means that ex post facto accident analysis of human performance is inaccurate.¹

1 . Richard I. Cook，复杂系统如何失败（认知技术实验室，2000 年）。

1. Richard I. Cook, How Complex Systems Fail (Cognitive Technologies Laboratory, 2000).

构建和管理大型复杂系统的组织需要专注于发现和解决问题，而不是仅仅找出根本原因，然后解决问题，包括回到最初推动复杂性产生的业务驱动因素。解决复杂性带来的持续失败的一种可能的解决方案是记住复杂性是解决难题的副作用。有时，唯一真正的解决方案、真正降低复杂性的唯一方法就是降低问题的难度。

Instead of just seeking out a root cause and then fixing the blame, organizations that build and manage large complex systems need to focus on finding and fixing the problem—including reaching back into the business drivers that drove the creation of complexity in the first place. One possible solution to constant failure through complexity is to remember that complexity arises as side effect of solving hard problems. Sometimes the only real solution, the only way to truly reduce complexity, is to reduce the hardness of the problems.

工程技能和故障管理

Engineering Skills and Failure Management

对于单个工程师来说，什么技能最适合管理大型复杂系统中的故障？这个问题有三个答案：

For the individual engineer, what skills are best for managing failure in a large complex system? There are three answers to this question:

• 能够了解在任何特定设计或部署中反馈循环在何处以及如何形成。正反馈回路可能是控制平面故障领域中最具破坏性的力量。作为一名网络工程师多年，我几乎总能指出正反馈循环是几乎所有控制平面故障的根本原因（或主要根本原因）的一部分。

• The ability to see where and how feedback loops can form in any particular design or deployment. Positive feedback loops are probably the most destructive force in the world of control plane failures; in many years as a network engineer, I can almost always point to a positive feedback loop as part of the root cause—or the primary root cause—in virtually every control plane failure.

• “穿透”抽象并“看到”现实世界中存在的交互表面的能力。能够以多种不同的方式对这些交互表面进行建模可以帮助工程师真正理解手头的问题，并在复杂性权衡的范围内找到一个好的解决方案。或者简单地忽略确定给定权衡的问题只会压倒任何可能的解决方案。

• The ability to “reach through” abstractions, and to “see” the interaction surfaces, as they exist in the real world. Being able to model these interaction surfaces in a number of different ways can help the engineer to really understand the problem at hand, and to find a good solution within the realm of complexity tradeoffs. Or to simply ignore the problem on determining the tradeoffs given will simply overwhelm any possible solution.

• 获得失败的经验。许多技能，例如骑自行车，必须通过身体经验来学习。管理大型复杂系统的故障是其中一项技能，就像骑自行车一样，它也是一项一旦学得好就不容易忘记的技能。在现实世界中，除了让新工程师处理较小的故障、教他们培养故障管理技能所需的思维和工程意识，然后让他们练习之外，没有简单的方法来培养这些技能。对于那些依赖大型复杂系统来学习的人来说，这是一个惨痛的教训，但却是必要的。

• Gaining experience with failure. Many skills, like riding a bicycle, must be learned through physical experience. Managing failure on large complex systems is one of those skills—and like riding a bike, it’s also a skill not easily forgotten once well learned. There is no simple way to develop these sorts of skills in the real world other than putting new engineers on smaller failures, teaching them the thinking and engineering sense needed to develop failure management skills, and then letting them practice. This is a hard lesson for those who depend on large complex systems to learn, but it’s a necessary one.

为什么不直接解决这些问题呢？因为复杂性不允许有“灵丹妙药”。图灵曲线总是能赶上任何提出的解决方案，无论考虑得多么周全。即使是编织得最紧密的网，意想不到的后果和漏洞抽象也总能找到办法潜入。

Why not just solve these problems? Because complexity doesn’t allow for a “silver bullet.” The Turing Curve always catches up with any proposed solution, no matter how well thought out. Unintended consequences and leaky abstractions will always find a way to sneak through even the most tightly woven net.

第 9 章可编程网络

Chapter 9. Programmable Networks

从某种意义上说，网络从一开始就是可编程的。BGP 社区最初由托尼·李 (Tony Li) 和雅科夫·雷克特 (Yakov Rekhter) 在华盛顿特区的一家酒吧绘制的两张餐巾纸上勾画出来，在路由控制平面内承载着复杂的政策。虽然许多路由协议（包括 OSPF、IS-IS 和 EIGRP）中的标记功能对于简单任务非常有用，但将完整策略集附加到单个前缀的能力增加了整个范围的功能。除了对网络进行编程之外，还可以考虑基于策略的路由和流量工程吗？现代网络的可编程性（截至发稿时，网络技术的变化几乎与男士领带的宽度和裙摆的长度一样快）有什么不同？DevOps 和软件定义网络 (SDN) 有何不同，是什么推动了可编程网络的发展？这个问题有几种可能的答案：

In a sense, networks have been programmable from the beginning. BGP communities, originally outlined on two napkins drawn at a Washington DC bar by Tony Li and Yakov Rekhter, enshrined carrying complex policy within a routed control plane. While the tagging capability in many routing protocols, including OSPF, IS-IS, and EIGRP were useful for simple tasks, the ability to attach a full policy set to a single prefix added an entire range of capabilities. What else can policy-based routing and traffic engineering be considered other than programming the network? What’s different with the modern drive to make networks programmable (at press time—networking technology changes almost as quickly as the width of men’s ties and the length of hemlines)? How are DevOps and software-defined networks (SDNs) different, and what’s driving this movement toward a programmable network? There are several possible answers to this question:

• 用于与控制和转发系统交互的特定机制。

• The specific mechanism used to interact with the control and forwarding systems.

• 业务变化率。

• The rate of business change.

• 技术的感知商业价值不断变化。

• The changing perceived business value of technology.

• 分布式控制平面的复杂性。

• The perceived complexity of distributed control planes.

本章为第 10 章“可编程网络复杂性”奠定了基础。它从驱动程序和定义开始，因为这些是网络工程师在考虑如何以及在何处部署网络可编程性时进行权衡的基础。第二部分考虑用例网络可编程性为业务驱动程序和定义增添了一些实质内容。然后，第三部分通过检查几个建议的接口来考虑 SDN 前景。

This chapter provides the groundwork for Chapter 10, “Programmable Network Complexity.” It begins with the drivers and a definition, as these underlie the tradeoffs network engineers make when considering how and where to deploy network programmability. The second section considers use cases for network programmability to put a little flesh on the bones of the business drivers and definition. The third section then considers the SDN landscape by examining several proposed interfaces.

驱动因素和定义

Drivers and Definition

几年前，一家国际服务提供商遭遇了大规模的网络中断。作为回应，该提供商的管理层要求 16 名知名网络设计和升级工程师（来自全球升级团队的杰出工程师和工程师）聚集在一个办公室，“尽其所能”工作，重新设计他们的网络，以便永远不会再失败。正如一位参与其中的工程师所说：“需要很长时间才能提出如此完美的设计。我们要做的是一个人在白板上写字，十五个人擦除。” 这个故事直接转移到定义可编程网络——如果你把十六个工程师放在一个房间里并要求他们定义“可编程网络”，你最终会得到一次写入和十五次擦除。

A number of years ago, an international service provider suffered a large-scale network outage. In response, the provider’s management demanded that 16 well-known network design and escalation engineers—distinguished engineers and engineers from the global escalation team—gather in a single office and work “as long as it took,” to redesign their network so it would never fail again. As one of the engineers involved remarked: “It’s going to take a long time to come up with such a perfect design. What we’re going to have is one person writing on the white board, and fifteen erasing.” This story transfers directly to defining the programmable network—if you put sixteen engineers in a room and ask them to define “the programmable network,” you’ll end up with one writing and fifteen erasing.

然而，由于您无法擦除本书的页面，因此本节将尝试回答以下问题：什么是可编程网络？——尽管在完成本节之前您可能会想在计算机屏幕上使用修正带。定义的最简单途径是通过可编程网络的驱动程序。这里将讨论驱动程序的两个子集：

Because you can’t erase the pages of this book, however, this section will attempt to answer the question, what is a programmable network?—though you might be tempted to use correction tape on your computer screen before this section is done. The easiest path to a definition is through the drivers for programmable networks. Two subsets of drivers will be discussed here:

• 业务驱动因素

• Business drivers

• 集权化的潮起潮落

• The ebb and flow of centralization

业务驱动因素

Business Drivers

每项业务都是信息业务。

Every business is an information business.

如果大多数企业认为自己不从事信息业务，那并不重要，因为他们确实从事信息业务。事实上，业务一直与信息有关，尽管多年来信息的焦点已经发生了变化。首先，有关于技术（手工艺）的信息，然后是关于谁可以在哪里进行贸易（商业系统）值得信赖的信息，然后是技术的回归（制造系统），等等。在当今世界，每个人脑海中浮现的信息都是关于客户的——他们是谁，他们想要什么，以及我如何以一种能引起他们注意的方式与他们“交谈”？

It doesn’t matter if most businesses believe they’re not in the information business—they are. In fact, business has always been about information, though the focus of the information has shifted over the years. First, there was information about technique (craftsmanship), then about who could be trusted where for trade (mercantile systems), then there was a return to technique (manufacturing systems), and so on. In today’s world, the information that rises to the top of everyone’s mind is about the customer—who are they, what do they want, and how do I “talk” to them in a way that will get their attention?

旧信息经济与新信息经济之间的一个关键区别在于企业领导者想要了解的事物变化的速度。制造系统、技术、配方、流程和贸易伙伴的变化比最新时尚要慢；客户的愿望就像暴风雨中的风一样改变方向。随着推动业务发展的信息速度不断加快，企业领导者自然会寻找管理这些信息的方法。正如吉尔·戴奇 (Jill Dyche) 指出的那样：“随着大数据等趋势的出现，商界人士更倾向于通过信息的视角来看待联系，整合关键数据，以形成单一的业务视图。” ¹在一组相当静态的系统（包括网络）上处理和生成信息非常困难。

A key difference between the old information economy and the new is the speed at which the things business leaders want to know is changing. Manufacturing systems, techniques, formulas, processes, and trading partners change more slowly than the latest fad; customer desires change direction like the wind in a storm. As the information driving the business increases in speed, it’s only natural for business leaders to look for ways to manage that information. As Jill Dyche points out: “With the advent of trends like big data, businesspeople are more apt to see connection through the lens of information, consolidating key data in an effort to reach a single view of the business.”¹ Handling and producing information on a fairly static set of systems (including the network) is very difficult.

1 . Jill Dyche，《新 IT：技术领导者如何在数字时代实现业务战略》（McGraw-Hill，2015 年），np

1. Jill Dyche, The New IT: How Technology Leaders Are Enabling Business Strategy in the Digital Age (McGraw-Hill, 2015), n.p.

虚拟拓扑可以以更快的方式快速有效地构建在物理底层之上，以响应新的应用程序和要求。虽然底层在设计和范围上可以保持相当恒定（通常通过使用横向扩展范式来实现，允许相当快地添加新容量），但可以为不同的应用程序或业务组启动和管理新的虚拟拓扑，以满足短期或长期的需求当术语需求出现时。

Virtual topologies can be built quickly and efficiently on top of a physical underlay in a much faster way to respond to new applications and requirements. While the underlay can remain fairly constant in design and scope (generally achieved by using a scale out paradigm allowing new capacity to be added fairly quickly), new virtual topologies can be spun up and managed for different applications or business groups to meet short or long term needs as they arise.

对于传统的分布式控制平面来说，定期启动此类覆盖是很困难的。可编程网络允许更快地构建和管理覆盖网络，从而提供以业务速度管理信息的能力。

Spinning up such overlays on a regular basis is difficult with traditional distributed control planes. Programmable networks allow overlays to be built and managed more quickly, providing the ability to manage information at the speed of business.

然而，推动收入增长（增加收入）只是可编程网络等式的一方面。手动配置网络需要大量时间、精力和专业知识，并且经常会因人为错误而导致许多错误。手动在大量设备上重复相同的配置将不可避免地导致某些地方出现错误。然而，大规模重复常见任务对于自动化来说已经成熟，可以节省时间，并通过减少 MTBM 来提高网络的可用性。自动化这些任务可以通过减少运行网络所需的运营支出来降低利润。

Driving the top line up—increasing revenues—is only one side of the equation for programmable networks, however. Manually provisioning networks requires a lot of time, effort, and expertise—and can often lead to a lot of mistakes just through human error. Repeating the same configuration across a large number of devices manually will inevitably result in some mistake being made someplace. Large-scale repetition of common tasks is, however, ripe for automation, saving time, and making the network more available by reducing the MTBM. Automating these tasks can drive the bottom line down by reducing the OPEX required to run the network.

可编程网络降低利润的另一种方法是提高网络的整体利用率。例如，流量通常遵循每日、每小时、每周和季节性模式。由于将流量快速转移到利用率较低的链路非常困难，网络运营商经常过度建设其网络。选择带宽以支持单个链路上的最高流量。在非高峰时段，该带宽不会被使用，因此是一种没有投资回报的成本。然而，如果网络是可编程的，则可以近乎实时地设计流量以适应网络上不断增加的负载，或者网络可以向某些应用程序发出信号（例如备份作业）等待另一个时间发送任何流量。以这种方式提前规划带宽使用情况称为带宽日历。虽然日历不能取代有效带宽规划的需要，但它可以将重点从构建峰值负载转移到构建更接近较长时间段内平均负载的数字。

Another way programmable networks can drive the bottom line down is by increasing the overall utilization of the network. For instance, traffic often follows daily, hourly, weekly, and seasonal patterns. Because of the difficulty of rapidly moving traffic around to lower utilized links, network operators often overbuild their networks. Bandwidth is chosen to support the highest traffic across a single link. During non-peak times, this bandwidth is not used, and is therefore a cost that has no return on investment. If the network is programmable, however, traffic can be engineered in near real time to adjust to increasing load on the network—or the network can signal some applications (such as a backup job) to wait until another time to send any traffic at all. Planning bandwidth usage in advance in this way is called bandwidth calendaring. While calendaring cannot replace the need for effective bandwidth planning, it can move the emphasis away from building for peak load and toward building for a number closer to the average load across longer time periods.

最后，许多网络运营商将可编程网络视为将硬件投资与软件或系统投资分开的一种方式。在传统网络中，控制平面以及网络的基本架构与硬件密切相关。购买特定供应商的设备会将运营商与特定的功能集联系起来。供应商经常以“足够标准”来打造精美艺术，以声称具有互操作性，同时包含可产生尽可能最高程度锁定的功能。从供应商的角度来看，这些“锁定功能”实际上是创新和附加值的例子。

Finally, many network operators see programmable networks as a way to separate their hardware investment from their software, or systems, investment. In traditional networking, the control plane—and therefore the basic architecture of the network—is intimately tied to the hardware. Purchasing a particular vendor’s equipment ties the operator to a particular feature set. Vendors have often made a fine art out of being “just standard enough” to claim interoperability, while including features that produce the highest degree of lock in possible. From the vendor’s perspective, these “lock in features” are actually examples of innovation and added value.

许多运营商将可编程网络视为摆脱与供应商之间这种反复纠缠的一种方式。可编程网络通常能够提供与供应商特定的“锁定功能”相同水平的服务，同时避免实际的供应商锁定。这反过来又可能降低资本支出（CAPEX），供应商被迫分别在硬件和软件功能上进行竞争。

Programmable networks are seen, by many operators, as a way out of this back and forth with vendors. A programmable network might often be able to provide the same level of service as vendor-specific “lock in features,” while avoiding the actual vendor lock in. This could, in turn, drive down Capital Expenditures (CAPEX), and vendors are forced to compete on hardware and software features separately.

中心化的潮起潮落

The Ebb and Flow of Centralization

当然，对可编程性的热潮部分是信息技术世界中集中化与分散化的正常潮起潮落的一部分。第一批台式计算机被塞进壁橱里，对 IT 部门来说是隐藏的。数据首先通过终端仿真卡连接到大型机的台式计算机上的屏幕抓取器从大型机拉入 Lotus 123 和其他软件包。随着数据在整个组织中分布，IT 试图通过迷你、结构化查询语言 (SQL) 和中间件重新获得控制权。截至发稿时，集中化再次风靡，这次是云计算。是什么驱动了这些周期？表 9.1讨论了一些。

Part of the rush to programmability is, of course, part of the normal ebb and flow of centralization versus decentralization in the world of information technology. The first desktop computers were stuffed into closets, hidden from the IT department. Data was first pulled off mainframes into Lotus 123 and other software packages through screen scrapers on desktop computers connected to the mainframe through terminal emulation cards. As data was distributed through the organization, IT tried to regain control through minis, Structured Query Language (SQL), and middleware. At press time, centralization is the rage again, this time with cloud computing. What drives these cycles? Table 9.1 discusses a few.

表 9.1 一般处理的集中式与分散式

Table 9.1 Centralized versus Decentralized for General Processing

对于通用计算来说，集中化和分散化的潮起潮落是显而易见的，但是网络控制平面呢？回顾网络的历史，可以观察到同样的循环。电话网络是“原始”网络，由一小群人操作，他们了解如何布线以进行手动交叉连接，并使用旋转继电器管理中央办公室，旋转继电器将根据顺序电路中断建立连接。随着时间的推移，这些集中管理系统实现了自动化，形成了大规模的专用小交换系统。

The ebb and flow of centralization and decentralization is obvious enough for general computing, but what about network control planes? Looking back across the history of networks, the same sort of cycle can be observed. Telephone networks were the “original” networks, operated by a small army of people who understood how to wire cables for manual cross connects, and to manage central offices with rotary relays that would make connections based on sequential circuit interruptions. These centralized management systems were automated over time, resulting in large-scale Private Branch Exchange systems.

公共电话网络最终陷入了与分组交换网络的竞争，采用了信令系统 7 中 IP 和其他分组交换网络中使用的分布式控制平面的原理。分布式网络最终接管了电话网络的角色，大部分语音进行IP 之上，由分布式控制平面推动。对可编程网络的推动也可以被视为再次对集中控制的推动。

The public telephone network eventually fell into competition with packet switched networks, adapting principles of the distributed control planes used in IP and other packet switched networks in Signaling System 7. Distributed networks eventually took over the role of the telephone network, with most voice carried on top of IP, facilitated by distributed control planes. The drive toward programmable networks can also be seen as a drive toward centralized control once again.

表 9.1考虑了中央存储库与分散计算和存储设备之间信息潮起潮落的一些原因。是什么推动了控制平面的集中和分散的潮起潮落？也许最强大的驱动因素是感知的复杂性。

Table 9.1 considered some of the reasons for the ebb and flow of information between central repositories and decentralized compute and storage devices. What drives the ebb and flow of centralization and decentralization in the control plane? Perhaps the strongest driver is perceived complexity.

当分布式系统成为常态时，网络工程师与这些分布式系统固有的复杂性建立了密切的个人关系。由于国外在中心化系统方面的经验很少，因此围绕如何通过中心化构建更简单系统的理论如雨后春笋般涌现，并出现了通过中心化来简化网络的趋势。在中心化为规则、去中心化为例外的时期，情况恰恰相反——去中心化似乎比目前使用的中心化混乱要简单得多，因此市场朝着去中心化系统的简单性方向发展。

When distributed systems are the norm, network engineers develop a close and personal relationship with the complexity inherent in those distributed systems. Because there is little experience abroad with centralized systems, theories around how to build a simpler system through centralization grow like mushrooms, and there’s a move toward centralization to simplify networks. In periods where centralization is the rule, and decentralization the exception, the opposite occurs—decentralization appears much simpler than the centralized mess currently in use, and so the market moves toward the perceived simplicity of a decentralized system.

当然，复杂性并不总是只是被感知到的，为了构建支持数据和处理集中化所需的大型网络，需要某种形式的网络可编程性。如此庞大的网络规模需要自动化流程来配置和管理数千个物理设备及其连接，以及支持在单个基础设施上运行的大量客户或应用程序所需的虚拟机和虚拟网络覆盖。

Complexity isn’t always just perceived, of course—to build the large networks required to support the centralization of data and processing, some form of network programmability is required. The sheer scale of such large networks requires automated processes to configure and manage thousands of physical devices and their connectivity, along with the virtual machines and virtual network overlays required to support a large number of customers or applications running on a single infrastructure.

定义网络可编程性

Defining Network Programmability

那么，这些就是网络可编程性的驱动因素：

These, then, are the drivers for network programmability:

• 提高网络适应业务需求的能力，而业务需求在消费者驱动的世界中变化得更快。

• Increasing the ability of the network to change to business requirements, which are, in turn, changing more quickly in a consumer-driven world.

• 通过自动化许多原本需要人才和时间的流程，降低管理大型网络所需的运营成本。

• Decreasing the OPEX required to manage large-scale networks by automating many processes that would otherwise require talent and time.

• 通过将软件与硬件分离，降低购买和构建大规模网络所需的资本支出。这允许网络架构与供应商驱动的架构分离，或者更确切地说，与明确绑定到特定金属板的硬件设备模型分离。这有两个效果：第一种是简单地让硬件与硬件竞争，软件与软件竞争，将通常被视为一个系统的系统分解为独立竞争的单独部分。第二是允许硬件和软件作为单独的系统进行管理，每个系统都有自己的生命周期。在这种模型中，更换硬件不需要对网络运行方式进行新的设计或架构概念。

• Decreasing the CAPEX required to purchase and build large-scale networks by separating software from hardware. This allows the architecture of the network to be separated from vendor-driven architectures, or rather the appliance model of hardware explicitly tied to a particular piece of sheet metal. This has two effects; the first is simply to put hardware in competition with hardware and software in competition with software, reducing what is normally seen as a system into separate pieces that compete independently. The second is to allow the hardware and software to be managed as separate systems, each with their own lifecycles. Replacing hardware, in this model, doesn’t necessitate a new design or architectural conception of the way the network operates.

• 数据从分布式计算机转向更集中的计算和存储资源。

• The ebb of data from distributed computers and into more centralized compute and storage resources.

• 相信集中/分散栅栏的另一边的草更绿。

• The belief that the grass is greener on the other side of the centralized/decentralized fence.

• 如果我“在这里发明它”，它一定会更好，这一信念也经常以某种方式发挥作用。在某些情况下，成为“特殊的雪花”或打造独角兽的愿望常常驱使工程师改变周围的事物“只是为了与众不同”。

• The belief that if I “invent it here,” it must be better often plays into the equation in some way, as well. The desire to be a “special snowflake,” or to build a unicorn, often drives engineers to change things around “just to be different,” in some number of cases.

根据迄今为止审查的驱动程序，如何定义网络可编程性？虽然没有完美的定义，但一个好的工作定义可能是：

Based on the drivers reviewed up to this point, how can network programmability be defined? While there is no perfect definition, a good working definition might be:

当控制平面和数据平面提供允许通过机器可读数据驱动的 API 修改和监视网络状态的接口时，网络是可编程的。

A network is programmable when the control and data planes provide an interface that allows the state of the network to be modified and monitored through a machine readable data-driven API.

该定义的范围很广，几乎涵盖了允许跨网络设备阵列监视和修改状态的所有机制。需要注意以下几点：

The scope of this definition is wide, encompassing almost any mechanism that allows the monitoring and modification of state across an array of network devices. Several points should be noted:

• 命令行界面(CLI)、图形用户界面和其他为与人类交互而设计的界面不包括在该定义中。

• Command-line interfaces (CLIs), Graphical User Interfaces, and other interfaces designed for interaction with humans would not be included in this definition.

• 不包括标准化；尽管标准化很重要，但具有一组接口的单一供应商网络仍然符合资格。从网络设计和管理的角度来看，这种类型的解决方案可能并不理想（甚至不可接受）。供应商中立性可以通过可编程网络来实现，但这并不是一定的结果。另一方面，如果网络设备上没有可编程接口，供应商中立几乎不可能实现。

• Standardization is not included; while standardization is important, a single vendor network with a single set of interfaces would still qualify. This type of solution might not be ideal (or even acceptable) from the network design and management perspective. Vendor neutrality might be achieved through programmable networks, but it’s not a certain outcome. On the other hand, vendor neutrality is going to be almost impossible to achieve without programmable interfaces on networking devices.

• 状态既是控制平面又是数据平面。可以围绕可编程 API 的更改构建完整的分类法（有关此类分类法，请参阅《网络架构艺术》中的第 17 章“软件定义网络”，思科出版社，2013 年）。

• The state is both the control and the data plane. An entire taxonomy can be built around what the programmable API changes (for such a taxonomy, see Chapter 17, “Software Defined Networks,” in The Art of Network Architecture, Cisco Press, 2013).

•管理网络和网络编程之间需要有所区别。信息流入和流出控制平面和数据平面的速度并不能可靠地衡量任何特定操作属于哪个类别。这里使用的定义是管理与设备相关，网络可编程性与控制或数据平面相关。这不是一个完美的分割，但它对于接下来的讨论很有用。

• There needs to be some differentiation between managing a network and programming a network. The speed at which information flows into and out of the control and data planes isn’t a reliable measure of which category any particular action belongs on. The definition used here is that management relates to devices, network programmability relates to the control or data plane. This isn’t a perfect split, but it’s useful for the discussions that follow.

笔记

Note

关于在此处的定义中包含 CLI 的屏幕抓取机制肯定存在一些问题。关于 CLI 需要注意两点。首先，屏幕抓取应用程序的复杂性比真正的机器界面的复杂性要大得多。屏幕抓取工具必须跟上每个设备的输出和输入格式的每一个可能的变化；处理每台设备的原始数据模型，即使它们都不同，并且可能会定期更改，也省去了一个步骤脱离处理。可以说，从屏幕抓取转向真正的可编程界面消除了流程中最复杂和最难维护的部分。其次，屏幕抓取是一种不完美的适应方法，适用于小规模问题，但并不能真正提供网络的整体视图，也不能以相同的方式进行扩展。由于这些原因，屏幕抓取并未作为可编程接口包含在此处 - 它将在这里或那里提到，但它不被认为是网络可编程性问题的未来经过深思熟虑的解决方案。

There is bound to be some question about the inclusion of screen scraping mechanisms off of a CLI in the definitions here. Two points should be noted about CLIs. First, the complexity of a screen scraping application is much greater than the complexity of a true machine interface. Screen scrapers must keep up with each possible change in the output and input formats of each device; dealing with the raw data models of each device, even if they are all different, and can potentially all change on a regular basis, removes one step out of the processing. It can be argued moving from screen scraping to a true programmable interface removes the most complex and difficult to maintain pieces of the process. Second, screen scraping is an imperfect adaption that works over small-scale problems, but doesn’t really provide a holistic view of the network, nor is it scalable in the same way. Screen scraping is not included here as a programmable interface for these reasons—it will be mentioned here and there, but it’s not considered a future looking well thought out solution for the network programmability problem.

可编程网络的用例

Use Cases for Programmable Networks

理解一项技术的一个好方法是思考它旨在解决的问题。检查用例不仅可以帮助工程师了解技术的工作原理，还可以帮助他们了解其工作原理的原因。这里介绍两个用例：带宽日历和软件定义的边界。

One good way to understand a technology is to think through the problems it’s designed to solve. Examining use cases not only helps engineers to understand how the technology works, but also to understand why it works the way it does. Two use cases are presented here: bandwidth calendaring and software-defined perimeter.

带宽日历

Bandwidth Calendaring

每个网络在不同时期都有一个正常的“使用模式”。有时网络订阅量过多，导致数据包丢失，有时网络订阅量不足，可用带宽未被使用。图 9.1说明了企业网络的典型带宽利用率。

Each network has a normal “pattern of usage” across time. There are times when the network is oversubscribed, and dropping packets, and other times when the network is undersubscribed, with bandwidth available that’s not being used. Figure 9.1 illustrates somewhat typical bandwidth utilization for a corporate network.

图 9.1 随时间变化的带宽利用率

Figure 9.1 Bandwidth Utilization across Time

显而易见的是网络（或链路、结构或测量利用率的其他点）非常低的时间。尝试将使用大量带宽的应用程序放入这些较低利用率的时段是很直观的。该领域的解决方案从简单到复杂都有。例如：

What should be immediately obvious are the times during which the network (or link, or fabric, or some other point at which this utilization is being measured) is very low. It’s intuitive to try and put applications that use a lot of bandwidth into these lower utilization periods. The solutions in this space range from the simple to the complex. For instance:

• 网络的利用率有多少？测量单个链路的利用率与测量网络中一组链路的利用率不同，特别是当任何应用程序可以使用任何特定路径，并且该路径实际上可能随时间而变化时（考虑到相同成本的负载共享、拓扑变化、 ETC。）。

• How much of the network is utilized? Measuring the utilization of a single link is different than measuring the utilization of a set of links across the network, especially when any application could use any particular path, and the path might actually change over time (given equal cost load sharing, topology changes, etc.).

• 应用程序启动大规模传输的时间必须手动管理。如果网络利用率发生变化，则时间也必须更改。虽然这个手动过程可以通过屏幕抓取和其他方式实现自动化，但这样的过程并不是理想的。

• The time at which the application kicks off large-scale transfers must be manually managed. If network utilization changes, then the times must be changed, as well. While this manual process can be automated through screen scraping and other means, no such process is going to be ideal.

• 如果网络利用率的间隙持续时间太短，应用程序可能无法在可用的利用率间隙内完成其工作。在这种情况下，应用程序是否应该以某种方式暂停，并且应该如何进行“暂停”？这种暂停对数据一致性和应用程序使用有什么影响？显而易见的解决方案是某种网络接口，包括反馈循环，使应用程序能够近乎实时地查看网络条件中数据流变化的影响。

• If the gaps in the network’s utilization are too short in duration, the application might not be able to finish its work within the available gap in utilization. In this case, should the application somehow be paused—and how should that “pausing” take place? What is the impact of such a pause on data consistency and application usage? The obvious solution is some sort of interface into the network, including a feedback loop that allows the application to see the impact of changes in the data flow in network conditions in near real time.

• 在某些情况下，应用程序流量可能会被放置在一组特定的链路上，从而使网络的其余部分不受应用程序流量的影响。这种类型的流量工程解决方案与带宽日历无关（这两种解决方案可以在单个网络中交互和重叠），而且也与跨单个结构的大象和老鼠流量无关。

• There may be situations where the applications traffic can be placed on a specific set of links, allowing the remainder of the network to be unaffected by the application’s traffic. This type of traffic engineering solution is tangential to bandwidth calendaring—the two solutions can interact and overlap in a single network—but also to elephant and mouse flows across a single fabric.

通过手动配置，可以解决这些挑战（需要许多人花费数年的时间在键盘上使用大量手指）。鉴于每个设备的用户界面不会经常改变，通过更先进的屏幕抓取类型的自动化，可以在该过程中做出明确的改进。然而，要真正使网络和应用程序之间的交互良好进行，网络需要一个可编程接口，应用程序和网络可以通过该接口进行通信。图 9.2说明了该用例。

These challenges can be met—with a lot of fingers on keyboards across many people-years of time—through manual configuration. A definite improvement can be made in the process through more advanced screen scraping types of automation, given the user interfaces of each device doesn’t change too often. To truly make the interaction between the network and application to work well, however, the network needs a programmable interface through which the application and the network can communicate. Figure 9.2 illustrates the use case.

图 9.2 带宽日历

Figure 9.2 Bandwidth Calendaring

流程如图9.2所示：

The process shown in Figure 9.2:

1.应用程序向控制器发送有关即将到来的流的通知。这可能包括有关流的大小和速率、流必须完成的时间、流暂停的能力以及流是否接近实时（流信息）或相当静态的数据（块）的信息。贮存）。

1. The application sends a notification to the controller about an upcoming flow. This could include information about the size and rate of the flow, the time by which the flow must be completed, the ability of the flow to be paused, and whether the flow is near real time (streaming information) or fairly static data (block storage).

2.网络中的转发设备提供有关当前利用率、队列大小和其他网络状态信息的信息。

2. The forwarding devices in the network provide information on the current utilization, queue sizes, and other network state information.

3.控制器与数据存储交互，该数据存储包含有关网络状态的历史信息，包括带宽利用率、队列深度、每个链路上的抖动和延迟等。

3. The controller interacts with a data store that contains historical information about the state of the network, including bandwidth utilization, queue depths, jitter and delay across each link, and so on.

4.控制器确定应用程序应在所需时间内开始完成的时间，同时对网络上运行的其他应用程序的影响最小。

4. The controller determines when the application should start to finish within the time required with minimal impact on the other applications running on the network.

5.控制器向应用程序发出何时应启动流程的信号。

5. The controller signals the application when it should kick the flow off.

6.如有必要，控制器将配置服务质量 (QoS)、保留链路、将流量移出链路等，以使应用程序成功传输。

6. If necessary, the controller will configure—Quality of Service (QoS), reserve links, move traffic off of links, and so on, to make the application’s transfer successfully.

7.如果网络条件动态变化，控制器可以重新计算在网络上传输应用程序流所需的时间，向应用程序发出信号以根据需要暂停流。

7. If network conditions change on the fly, the controller can recalculate the time required to carry the application’s flow across the network, signaling the application to pause the flow as needed.

这种类型的过程需要转发设备、应用程序和控制器之间有一组可靠的接口。如果没有这样一组接口，这一切可以说是可能的，但很难大规模实现。

This type of process requires a solid set of interfaces between the forwarding devices, the application, and a controller. This is all arguably possible without such a set of interfaces, but it would be difficult to achieve on a large scale.

为了进一步实现这一目标，任何此类接口都应该是开放的，任何应用程序，无论是内置的还是商业的，都应该能够与任何控制器进行通信，并且控制器应该能够与任何一组网络进行通信设备，无论供应商如何。最后一个含义是否仅适用于硬件（任何供应商的硬件与单一供应商的软件），还是同时适用于软件和硬件，这是计算机网络世界中进一步讨论的问题（实际上是政治策略和激烈的争论）。

To carry this one step further, any such interface should be open, in the sense that any application, either built in house or commercial, should be able to communicate with any controller, and the controller should be able to communicate with any set of network devices, regardless of the vendor. Whether this last implication applies only to the hardware—any vendor’s hardware with a single vendor’s software—or across both software and hardware is a matter of further discussion (actually political maneuvers and loud boisterous arguments) within the computer networking world.

软件定义的边界

Software-Defined Perimeter

网络安全通常被定义为一座城堡；有内部和外部，如图9.3所示。

Network security is often defined like a castle; there is an inside and an outside, as illustrated in Figure 9.3.

图 9.3 以城堡安全为例

Figure 9.3 Castle Security as an Example

在图9.3中，有城墙防御系统的一部分。

In Figure 9.3, there is a portion of a castle wall defense system.

• 端塔A 和E 为城墙和大门提供覆盖。防御者可以从这些点向任何攻击者的侧翼射击，阻止对城墙和大门系统的攻击。

• The end towers, A and E, provide coverage for the wall and gate. Defenders can shoot into the flank of any attackers from these points, discouraging attacks on the walls and gate system.

• 外门B 并不是您想象中的堡垒——尽管它们在电影中看起来往往令人印象深刻。事实上，城墙的城门通常是系统，而不是单一的门。在古代以色列，该系统有六个房间和七个门，其中最里面的门是最强大的。B门当然是一扇强大的门，但它的主要目的是过滤，而不是完全封锁。如果攻击者能够通过这扇门，它会设计有一个从上方落下的辅助门（一个吊闸），将攻击者困在第一个房间内，在那里他们可以在周围墙壁和天花板的保护下被摧毁。将敌人分成更小的群体（分而治之）是墙系统的主要策略。

• The outer gate, B is not the bulwark you might think—though they tend to look impressive in movies. In reality, the gate of a walled city was normally a system, rather than a single gate. In ancient Israel, the system has six chambers with seven gates, with the innermost gate being the most formidable. Gate B is a formidable gate, of course, but its primary purpose is to filter, rather than to completely block. If attackers can get past this gate, it is designed with a secondary gate that will drop from above—a portcullis—trapping attackers inside the first chamber, where they can be destroyed from the protection of the surrounding walls and ceilings. Breaking your enemy up into smaller groups (divide and conquer) was much of the strategy of wall systems.

• C 和D 是内门。每个房间都是一个完整的防御系统，被设计为万一前一个大门失效时的后退——纵深防御的第一阶段。从某种意义上说，C 门只是为了保护 D 门免受攻击（保护内门的完整性）。

• C and D are inner gates. Each chamber is a complete defense system, designed as a fall back in case the previous gate fails—the first stage of defense in depth. In a sense, gate C is only there to protect gate D from attack (to protect the integrity of the inner gate).

如果入侵者进入外墙，通常会有一座带门的内墙，等等，最后是一个要塞，或某种形式的寺庙，作为堡垒中的完整堡垒。这种墙体系统在当时很有效。250 名守军可以抵御规模惊人的军队，守住一座有城墙的城市，因此需要长期围攻才能饿死守军。对有城墙的城市最有效的攻击是内部攻击——贿赂城里的某人以打开较小的侧门。在现代世界，这些系统看起来令人印象深刻，但效果不佳。为什么？

If an invader made it inside the outer wall, there was often an inner wall with gates, and so on, and finally a keep, or some form of temple that would serve as a complete fortress within a fortress. Such wall systems were effective in their time; 250 defenders could hold a walled city against armies of incredible sizes, necessitating long sieges to starve the defenders out. The most effective attack against a walled city was an insider attack—bribing someone in the city to leave a smaller side gate open. In the modern world, these systems seem impressive but ineffectual. Why?

• 大炮使攻破城墙变得更加容易。

• Artillery made breaching the walls much easier.

• 隧道机制允许从下面突破墙壁。

• Tunneling mechanisms allowed breaching the walls from underneath.

• 飞机使得飞越墙壁成为可能，从上面扔下东西。

• Airplanes made it possible to just fly over the walls, dropping stuff from above.

网络安全边界的构建很像一座城墙，如图9.4所示：

Network security perimeters are built much like a walled city, as shown in Figure 9.4:

• 路由器F 主要配置为执行一些基本过滤。这并不是真正要阻止大多数攻击，而是为了保护G（防火墙）免受外部通过旨在溢出链接、G的输入接口或其他方式的攻击。这相当于城墙防御系统上的B门；该外层可以快速处理一些简单的攻击，而无需使用大量网络资源。

• Router F is configured primarily to do some basic filtering. This isn’t really to block most attacks, but rather to protect G, a firewall, from attack from the outside through methods designed to overflow the link, G’s input interface, or other means. This is the equivalent of gate B on the walled defense system; this outer layer takes care of some simple attacks quickly without using a lot of network resources.

• 主机H 可能托管“牺牲”服务，但它也充当蜜罐。任何针对此服务器的攻击都可以测量和量化，这有助于在路由器 F 上构建更好的过滤器，在路由器 G 上构建更好的状态检查规则。此处收集的信息还可以指示攻击者试图访问哪些内部服务，等等。这相当于城墙防御系统中B门和C门之间的密室。请注意，如果该主机被接管，则可以用作入侵者的攻击基地，就像城墙城门之间的房间一样。这样的主机需要受到严格的本地安全控制。

• Host H may host “sacrificial” services, but it also serves as a honeypot. Any attacks against this server can be measured and quantified, which helps build better filters on Router F and better stateful inspection rules on G. Information gathered here can also indicate which internal services attackers are trying to reach, and so on. This is the equivalent of the chamber between Gates B and C in the walled defense system. Note this host can be used as a base of attack by an intruder if it is taken over, much as the chamber between the gates in a walled city. Such a host needs to be well controlled with tight local security.

• 安全设备G 可能包括无状态过滤器、有状态数据包过滤器和网络地址转换。这是“主门”，类似于城墙防御系统中的 C 门。此时网络地址转换提供了“故障关闭”保障；软件错误会导致连接丢失，而不是开放访问。

• The security appliance, G, may include stateless filters, stateful packet filters, and network address translation. This is the “primary gate,” similar to gate C in the walled defense system. Network address translation at this point provides a “fail closed” safeguard; software errors result in a loss of connectivity, rather than open access.

图 9.4 网络非军事区 (DMZ)

Figure 9.4 A Network Demilitarized Zone (DMZ)

传统DMZ系统的缺点是什么？与城墙的城市几乎相同：

What are the weaknesses of the traditional DMZ system? Pretty much the same as a walled city:

• 拒绝服务(DoS) 攻击可以向DMZ 投掷大炮，破坏或破坏网络门系统。

• Denial of Service (DoS) attacks can throw artillery at the DMZ, breaking or breaching the network gate system.

• 通往外界的隧道提供了一个入口点，攻击者可以通过该入口点穿过墙壁而无需穿过大门。

• Tunnels to the outside world provide an entry point through which attackers can reach through the wall without passing through the gates.

• 直接针对服务器的攻击（例如针对 HTTP 服务器的脚本攻击）提供了一种“飞越”防御系统的方法。

• Direct to server attacks (such as scripting attacks against HTTP servers) provide a way to “fly over” the defense system.

• 当然，内部攻击始终存在。

• And, of course, there’s always the ever present insider attack.

事实证明，自中世纪以来，情况并没有太大改变。随着攻击网络围墙的工具变得更加先进，网络安全需要从构建围墙转向构建能够防御特定地点免受各种攻击的流动机动部队——就像现代流动机械化部队一样。用不太军事化的术语来说，网络需要从外部松脆、中间耐嚼转变为彻底松脆。

As it turns out, things haven’t changed much since the Middle Ages. As the tools for attacking the network’s walls become more advanced, network security needs to move from building walls to building fluid mobile forces that can defend specific spots against a variety of attacks—just like the modern fluid mechanized force. To put it in less military terms, the network needs to move from being crunchy on the outside and chewy in the middle to being crunchy through and through.

但网络如何实现转变呢？一种可能性是使用可编程网络中可用的工具。图 9.5说明了这一点。

But how do networks make the transition? One possibility is to enlist the tools available in the programmable network. Figure 9.5 illustrates.

图 9.5 软件定义的边界示例

Figure 9.5 Software-Defined Perimeter Example

在这个网络中：

In this network:

1.主机 A 向位于服务器 D 上的服务发送数据包。对于此示例，假设此数据包正在启动与服务的会话，并且数据包包含某种形式的服务登录凭据。

1. Host A sends a packet toward a service, located on Server D. For this example, assume this packet is initiating a session with the service, and the packet contains some form of login credentials to the service.

2.路由器 B 并没有简单地转发此数据包，而是注意到该服务已被标记为安全，因此它将数据包重定向到控制器。

2. Router B, rather than simply forwarding this packet, notes the service has been marked secure, so it redirects the packet toward a controller.

3.控制器 E 检查数据包并确定数据包的目标服务。然后，控制器联系服务本身（如图所示）或某些集中式身份服务（例如 OpenStack 中的 Keystone 服务实例），以验证用户访问服务的能力。

3. The controller, E, examines the packet and determines which service the packet is destined toward. The controller then contacts the service itself (shown here), or some centralized identity service (such as a Keystone service instance in OpenStack), to verify the user’s ability to access the service.

4.目标服务或身份服务返回拒绝该特定用户访问服务的请求。

4. The target service, or the identity service, returns a denial for the request to access the service for this particular user.

5.控制器使用此信息设置过滤器，阻止用户访问服务所在的服务器 D，并以拒绝或登录失败的方式回复原始数据包。

5. The controller uses this information to set up a filter blocking the user from accessing the server, D, on which the service resides, and replies to the original packet with a denial or login failed.

虽然这更多的是身份验证、授权、访问 (AAA) 示例，但相同的原则也适用于状态过滤和其他安全措施。可编程网络可以将安全策略推送到网络边缘，在每个服务或每个网络拓扑元素的基础上提供软件定义的边界。

While this is more of an Authentication, Authorization, Access (AAA) example, the same principle holds for stateful filtering and other security measures. Programmable networks can push security policy to the edge of the network, providing a software-defined perimeter on a per service or per network topology element basis.

可编程网络接口

Programmable Network Interfaces

接口是弥合传统控制平面、应用程序和策略实施与可编程网络之间差距的关键点。网络（或网络设备）与对网络进行编程的应用程序或系统之间的接口分为两个不同的部分。

Interfaces are a key point in bridging the gap between traditional control planes, applications and policy implementation, and the programmable network. The interface between the network (or network devices) and applications or systems programming the network is broken into two different pieces.

北向接口是各个网络设备向控制器提供的信息；这些信息通常会汇总到整个网络的更大视图中。该接口包含三种特定类型的信息：

The northbound interface is information individual network devices provide to controllers; this information would normally be rolled up into a larger view of the network as a whole. This interface contains three specific kinds of information:

•功能：提供可以编程或控制的信息。这包括元数据，或者更确切地说，有关如何组织控制结构以及如何通过网络设备向控制器呈现信息的信息。

• Capabilities: Provides information what can be programmed or controlled. This included metadata, or rather information about how the control structures are organized, and how information is presented by network devices to the controller.

•库存：提供有关网络中安装哪些设备的信息，可能包括有关物理连接的任何信息。

• Inventory: Provides information about what devices are installed where in the network, potentially including any information about physical connections.

•拓扑：提供有关连接网络设备的链路状态的信息。这可能包括本机带宽等内容。

• Topology: Provides information about the state of links connecting network devices. This may include things like the native bandwidth.

•遥测：包括操作状态、计数器以及有关当前网络状态的其他信息。这包括可用带宽、队列深度、延迟和抖动等。

• Telemetry: Includes operational state, counters, and other information about the current network state. This includes things like the available bandwidth, queue depths, delay, and jitter.

南向接口是从控制器推送到网络（或网络设备）的所需状态。这里至少有三个具体的接口：

The southbound interface is desired state pushed from the controller into the network (or network devices). There are at least three specific interfaces here:

•转发信息库(FIB)：用于通过交换设备转发数据包的实际表的直接接口。该接口绕过内部控制平面进程、静态配置的转发信息等。

• Forwarding Information Base (FIB): A direct interface into the actual tables used to forward packets through the switching device. This interface bypasses and internal control plane processes, statically configured forwarding information, and so on.

•路由信息库(RIB)：用于构建FIB 的表的直接接口。注入到该接口的信息与其他控制平面进程安装的转发信息、静态转发信息等混合在一起。这可能包括进入协议特定表的接口，例如内部 RIB 或拓扑表 - 然而，此类接口的问题是它们直接干扰分布式协议的操作，这些协议具有非常特定的规则，旨在防止最佳计算中的循环。路径。与这些表的接口可以有更大的通过注入导致协议无法收敛的信息，比单个设备故障或本地路由环路产生的副作用更大。

• Routing Information Base (RIB): A direct interface into the tables used to build the FIB. Information injected into this interface is mixed with forwarding information installed by other control plane process, static forwarding information, and so on. This could include interfaces into protocol specific tables, such as an internal RIB or topology table—the problem with such interfaces, however, is they interfere directly with the operation of distributed protocols that have very specific rules designed to prevent loops in the calculation of best paths. Interfacing with these tables can have much larger side effects than a single device failure or a local routing loop by injecting information that causes the protocol to fail to converge.

•交换路径：与转发表无关但直接影响数据包处理（例如QoS）的转发参数的接口。

• Switching Path: An interface into forwarding parameters that don’t relate to the forwarding tables, but directly impact packet handling, such as QoS.

这些接口都不包含传统上被认为是网络管理的内容。它们三者都与数据包通过设备时的处理方式直接相关，而不是与功率级别、任何已安装处理器的温度、内存利用率等相关。图 9.6说明了这些接口。

None of these interfaces encompass what might traditionally be considered network management. They all three relate directly to how packets are handled as they pass through the device, and not to things like the power level, the temperature of any installed processors, memory utilization, and so on. Figure 9.6 illustrates these interfaces.

图 9.6 可编程网络接口

Figure 9.6 Programmable Network Interfaces

可编程网络格局

The Programmable Network Landscape

正在开发哪些技术来解决迄今为止描述的用例？本节概述了可编程网络空间中开发（或正在开发）的四种截然不同的技术。此概述并未提供完整的图片，因为此处的重点是提供可靠的信息考虑网络可编程性和复杂性的背景。将考虑的技术包括 OpenFlow、YANG、路径计算元素协议 (PCEP) 和路由系统接口 (I2RS)。

What technologies are being developed to address the use cases described so far? This section provides an overview of four very different types of technology developed (or being developed) in the programmable network space. This overview doesn’t offer a complete picture, because the focus here is to provide a solid background for considering network programmability and complexity. The technologies that will be considered are OpenFlow, YANG, Path Computation Element Protocol (PCEP), and the Interface to the Routing System (I2RS).

开放流

OpenFlow

OpenFlow 最初旨在促进在实验环境中研究新的控制平面协议和系统。由于大多数高速转发硬件都与特定供应商的设备绑定，因此几乎没有空间来开发和实施新的控制平面来查看是否可以找到更有效的管理可达性广告的系统。OpenFlow 是在网络行业开发的，当时大型机箱式交换机很常见；在机箱交换机中，控制平面在路由处理器上运行，而实际的高速转发硬件则在线卡上安装和配置。这种功能分离提供了一个可以连接外部设备的自然点；在大多数基于机箱的路由器中，将路由处理器与线卡分开，允许在路由处理器上运行的控制平面进程被部署在外部设备（控制器）上的不同控制平面所取代。

OpenFlow was originally designed to facilitate the study of new control plane protocols and systems in an experimental setting. As most high speed forwarding hardware is tied to a specific vendor’s equipment, there is little room to develop and implement a new control plane to see if more efficient systems of managing the advertisement of reachability can be found. OpenFlow was developed at a time in the networking industry when large chassis-type switches were common; in a chassis switch the control plane runs on a route processor, while the actual high-speed forwarding hardware is installed and configured on line cards. This separation of functionality provides a natural point at which an external device can be connected; separating the route processor from the line cards in most chassis-based routers allows the control plane processes running on the route processor to be replaced by a different control plane deployed on an external device, a controller.

在图9.6中，OpenFlow是向FIB提供转发信息的南向接口。

Within Figure 9.6, then, OpenFlow is a southbound interface providing forwarding information to the FIB.

图 9.7说明了 OpenFlow 的操作。

Figure 9.7 illustrates the operation of OpenFlow.

图 9.7 OpenFlow 操作

Figure 9.7 OpenFlow Operation

在此图中：

In this figure:

1.主机 A 向服务器 D 发送数据包。

1. Host A transmits a packet toward Server D.

2. Router B 收到此数据包，检查其本地转发表，发现没有此数据包的转发信息目的地。由于本地没有转发信息，Router B 将报文转发给Controller E 进行处理。

2. This packet is received by Router B, which examines its local forwarding table, and discovers it doesn’t have forwarding information for this destination. As there is no local forwarding information, Router B forwards the packet to the Controller, E, for processing.

3.控制器 E 检查从路由器 B 收到的数据包。控制器确定正确的路由信息（发现此信息的过程由控制器决定，但它可以像连接控制器到网络中的每个路由器，因此它对每个目的地都有完全的可见性，或者控制器可以运行分布式路由协议来与其他控制器交换可达性信息）。一旦确定了这些信息（包括下一跳重写、正确的出站接口等），流条目就会安装到路由器 B 的转发表中。这将创建路由器 B 在此流中转发数据包所需的状态。

3. The Controller, E, examines the packet it has received from Router B. The controller determines the correct routing information (the process by which this information is discovered is left up to the controller, but it could be something as simple as connecting the controller to every router in the network, so it has full visibility for every destination—or the controller could run a distributed routing protocol to exchange reachability information with other controllers). Once this information is determined, including a next hop rewrite, the correct outbound interface, and so on, a flow entry is installed into Router B’s forwarding table. This creates the state necessary for Router B to forward packets in this flow.

4.控制器E 也可以确定该流量必须经过Router C，因此它也将计算出的出接口和其他信息安装到Router C 的转发表中。

4. The Controller, E, can also determine this traffic flow must pass through Router C, so it installs the calculated outbound interface and other information into Router C’s forwarding table, as well.

5.主机 A 将来向服务器 D 发送的一些数据包现在将根据此缓存的流标签信息通过路由器 B 和 C 转发。

5. Some future packet Host A sends toward Server D will now be forwarded through Routers B and C based on this cached flow label information.

笔记

Note

对于长期的网络工程师来说，这个数据包处理过程应该听起来很熟悉。例如，它与旧版思科路由器的“快速缓存”处理几乎相同；有关详细信息，请参阅Cisco IOS 软件架构内部。

This packet processing process should sound familiar to long time network engineers. It is almost identical to the “fast cache” processing of older Cisco routers, for instance; see Inside Cisco IOS Software Architecture for more information.

关于 OpenFlow 需要记住的一个关键点是，它最初设计为携带来自转发数据包标头的完整信息的流条目，包括源 IP 地址、目标 IP 地址、IP 协议号、源端口和目标港口。流标签可以包含将实现每个特定流的转发的特定信息，也可以通过通配符包含一组流的信息。例如，可以使用仅包含目标 IP 地址信息的流条目来模拟标准 IP 路由。

A key point to remember about OpenFlow is it was originally designed to carry flow entries with full information from the header of the forwarded packet, including the source IP address, the destination IP address, the IP protocol number, the source port, and the destination port. Flow labels can contain specific information that will implement forwarding for each specific flow, or they can contain information for a group of flows through wildcards. For instance, a standard IP route can be emulated using flow entries with just the destination IP address information.

OpenFlow 已被众多供应商不同程度地实施。

OpenFlow has been implemented, in varying degrees, by a wide array of vendors.

笔记

Note

有关 OpenFlow 的更完整描述，请参阅William Stallings 所著的《现代网络基础：SDN、NFV、QoS、IoT 和云》（Addison-Wesley Professional，2015 年）。

See Foundations of Modern Networking: SDN, NFV, QoS, IoT, and Cloud by William Stallings (Addison-Wesley Professional, 2015) for a more complete description of OpenFlow.

杨

YANG

在大多数协议中，数据模型绑定到协议本身。例如，IS-IS 协议在一组类型长度向量中携带可达性信息，并绑定在任何设备上运行的任何 IS-IS 进程都可以理解的特定数据包格式中。因此，协议中携带的信息的格式和信息的传输机制被组合成一个对象或一件事物。YANG 的不同之处在于它实际上是一种建模语言，旨在描述网络设备中的转发和其他状态。

In most protocols, the data model is bound into the protocol itself. For instance, the IS-IS protocol carries reachability information in a set of Type-Length-Vectors, bound within a specific packet format any IS-IS process running on any device can understand. Hence, the format of the information carried within the protocol and the transportation mechanism for the information are combined into one object, or one thing. YANG is different in that it is actually a modeling language designed to describe the forwarding and other state in network devices.

笔记

Note

数据模型和信息模型密切相关但又不同。信息模型描述信息流、信息结构以及处理信息的不同进程之间的交互。数据模型关注信息本身的结构，包括保存信息的不同结构之间的关系。YANG作为一种建模语言，是一种数据模型而不是信息模型，因为它关注的是信息的结构。基于 YANG 的信息模型可能包括使用 YANG 模型构建的数据传输，以及有关如何使用数据来实现网络中特定状态的信息。

Data models and information models are closely related but different things. An information model describes the flow of information as well as the structure of the information and the interaction between the different processes that handle the information. A data model is focused on the structure of the information itself, including the relationships between different structures that hold information. YANG, as a modeling language, is a data model rather than an information model, as it focuses on the structure of information. An information model based on YANG could potentially include the transport of data structured using a YANG model, as well as information about how the data is used to achieve specific states in the network.

笔记

Note

YANG 规范由互联网工程任务组 (IETF)发布为 RFC6020 ^{2 。}

The YANG specification is published as RFC6020² by the Internet Engineering Task Force (IETF).

2 . M. Bjorklund 编辑，“YANG - 网络配置协议 (NETCONF) 的数据建模语言”（IETF，2010 年 10 月），2015 年 9 月 24 日访问，https: //datatracker.ietf.org/doc/rfc6020/。

2. M. Bjorklund, ed., “YANG - A Data Modeling Language for the Network Configuration Protocol (NETCONF)” (IETF, October 2010), accessed September 24, 2015, https://datatracker.ietf.org/doc/rfc6020/.

YANG 是一种模块化语言，表示为可扩展标记语言 (XML) 树结构 — XML 的一个更常见的子集是 HTML，用于承载浏览器呈现为网页的指令。作为一种语言，YANG 并没有真正指定任何有关网络设备的信息；它只是提供了一个用于表达有关网络设备的信息的框架，就像任何自然语言的一组语法规则一样。

YANG is a modular language expressed as an eXtensible Markup Language (XML) tree structure—a more familiar subset of XML is the HTML, used to carry the instructions browsers render as web pages. As a language, YANG doesn’t really specify any information about network devices; it just provides a framework for expressing information about network devices, much like a set of grammar rules for any natural language.

笔记

Note

虽然 HTML 是 XML 的一个粗略子集，但 HTML 实际上是在 XML 之前开发的。HTML 在该领域的成功导致了超集标记系统的开发，该系统可以用于比 HTML 更广泛的用途，包括信息的一般结构，例如 YANG。从某种意义上说，HTML 是 XML 的“之父”，而 XML 又催生了许多 HTML 的同类。在这个意义上，YANG 可以被认为是 YANG 的对等体，因为它是用于特定情况的 YANG 的子集或更具体的定义。

While HTML is a rough subset of XML, HTML was actually developed before XML. The success of HTML in the field led to the development of a superset markup system that could be used for a wider array of uses than HTML, including the general structuring of information such as YANG. In a sense, HTML is the “father” of XML, which, in turn, has spawned a number of peers to HTML. YANG can be considered a peer of YANG in this sense, as it’s a subset or more specific definition of YANG used for a specific case.

在撰写本文时，正在为网络协议和设备开发许多 YANG 模型。例如，下面是用于构建链接失败通知的模型片段。³

A number of YANG models are being developed for network protocols and devices at the time of this writing. For instance, a snippet of a model for structuring the notification of a link failure follows.³

3 . M Bjorklund，“用于网络配置协议 (NETCONF) 的 YANG-A 数据建模语言”，YANG Central，np，最后修改于 2010 年 10 月，访问于 2015 年 6 月 14 日，http://www.yang-central.org/twiki/ pub/Main/YangDocuments/rfc6020.html#rfc.section.4.2.2.5。

3. M Bjorklund, “YANG-A Data Modeling Language for the Network Configuration Protocol (NETCONF),” YANG Central, n.p., last modified October 2010, accessed June 14, 2015, http://www.yang-central.org/twiki/pub/Main/YangDocuments/rfc6020.html#rfc.section.4.2.2.5.